deep-photo-styletransfer

Code and data for paper "Deep Photo Style Transfer"

Setup

This code is based on torch. It has been tested on Ubuntu 14.04 LTS.

Dependencies:

Torch (with matio-ffi and loadcaffe)
Matlab or Octave

CUDA backend:

CUDA
cudnn

Download VGG-19:

sh models/download_models.sh

Compile cuda_utils.cu (Adjust PREFIX and NVCC_PREFIX in makefile for your machine):

make clean && make

Usage

Quick start

To generate all results (in examples/) using the provided scripts, simply run

run('gen_laplacian/gen_laplacian.m')

in Matlab or Octave and then

python gen_all.py

in Python. The final output will be in examples/final_results/.

Basic usage

Given input and style images with semantic segmentation masks, put them in examples/ respectively. They will have the following filename form: examples/input/in<id>.png, examples/style/tar<id>.png and examples/segmentation/in<id>.png, examples/segmentation/tar<id>.png;
Compute the matting Laplacian matrix using gen_laplacian/gen_laplacian.m in Matlab. The output matrix will have the following filename form: gen_laplacian/Input_Laplacian_3x3_1e-7_CSR<id>.mat;
Run the following script to generate segmented intermediate result:

th neuralstyle_seg.lua -content_image <input> -style_image <style> -content_seg <inputMask> -style_seg <styleMask> -index <id> -serial <intermediate_folder>

Run the following script to generate final result:

th deepmatting_seg.lua -content_image <input> -style_image <style> -content_seg <inputMask> -style_seg <styleMask> -index <id> -init_image <intermediate_folder/out<id>_t_1000.png> -serial <final_folder> -f_radius 15 -f_edge 0.01

Note: In the main paper we generate all comparison results using automatic scene segmenation algorithm modified from DilatedNet. Manual segmentation enables more diverse tasks hence we provide the masks in examples/segmentation/.