This repository contains an application which converts the input from your webcam to an artistic equivalent.
This is implemented using real-time style transfer described in the paper by
Johnson et al. Perceptual Losses for Real-Time Style Transfer
and Super-Resolution.
The neural networks, contained in the networks
folder, were trained using the COCO dataset,
which was collected by the COCO Consortium (cocodataset.org).
Make sure you have the minimum following requirements:
- MATLAB R2020a or later
- Deep Learning Toolbox
- MATLAB Support Package for USB Webcams
- A webcam
- A supported GPU (optional for better performance)
Note that, once you have MATLAB installed, the easiest way to install toolboxes and support packages is the Add-On Explorer.
- Download or clone this repository to your machine.
- Open the repository in MATLAB.
- Connect a supported webcam if no built-in webcam is available.
- Open the project file
ArtisticStyleTransfer.prj
. This action adds the necessary folders to the path and opens the project view. - Right click on the
ArtisticStyleTransfer.mlapp
file and choose Run.
When you open the app, the pretrained networks in the networks
folder are loaded. This operation takes a few seconds to complete.
Once loading is completed, you should see something similar to the image below.
From this view, you can:
- Change the applied style using the dropdown menu or the left and right arrows.
- Take a picture. The images are saved in a folder called
pictures
. This folder is created in your current working directory. - Choose whether to use the CPU or the GPU for the image processing. If a GPU is not available, this option is disabled.
- Choose a resolution. Note that a low resolution results in less image processing and a more responsive app.
If you change the resolution, then you can notice that the size of the features also changes. This is because the pre-trained networks have learned features of a predefined size in terms of pixels. When you reduce the resolution, you are effectively giving a smaller image to the network. The app resizes the image to fill the screen. Therefore, if you set the resolution to low, the features appear larger within the image, even though the features are of the same size in terms of pixels. This is demonstrated in the image below.
Please file any bug reports or feature requests as GitHub issues.
Copyright 2020 The MathWorks, Inc.