Artistic Style Transfer with a Webcam

This repository contains an application which converts the input from your webcam to an artistic equivalent. This is implemented using real-time style transfer described in the paper by Johnson et al. Perceptual Losses for Real-Time Style Transfer and Super-Resolution. The neural networks, contained in the networks folder, were trained using the COCO dataset, which was collected by the COCO Consortium (cocodataset.org).

📦 Requirements
🏁 Getting Started
🖼️ Usage
❓ Resolution and Feature Size
💬 Contribute

Requirements

Make sure you have the minimum following requirements:

MATLAB R2020a or later
Deep Learning Toolbox
MATLAB Support Package for USB Webcams
A webcam
A supported GPU (optional for better performance)

Note that, once you have MATLAB installed, the easiest way to install toolboxes and support packages is the Add-On Explorer.

Getting Started

Download or clone this repository to your machine.
Open the repository in MATLAB.
Connect a supported webcam if no built-in webcam is available.
Open the project file ArtisticStyleTransfer.prj. This action adds the necessary folders to the path and opens the project view.
Right click on the ArtisticStyleTransfer.mlapp file and choose Run.

Usage

When you open the app, the pretrained networks in the networks folder are loaded. This operation takes a few seconds to complete. Once loading is completed, you should see something similar to the image below.

From this view, you can:

Change the applied style using the dropdown menu or the left and right arrows.
Take a picture. The images are saved in a folder called pictures. This folder is created in your current working directory.
Choose whether to use the CPU or the GPU for the image processing. If a GPU is not available, this option is disabled.
Choose a resolution. Note that a low resolution results in less image processing and a more responsive app.

Resolution and Feature Size

If you change the resolution, then you can notice that the size of the features also changes. This is because the pre-trained networks have learned features of a predefined size in terms of pixels. When you reduce the resolution, you are effectively giving a smaller image to the network. The app resizes the image to fill the screen. Therefore, if you set the resolution to low, the features appear larger within the image, even though the features are of the same size in terms of pixels. This is demonstrated in the image below.

Contribute

Please file any bug reports or feature requests as GitHub issues.