The major contributors of this repository include Jing Liao, Yuan Yao, Lu Yuan, Gang Hua and Sing Bing Kang at Microsoft Research.
Deep Image Analogy is a technique to find semantically-meaningful dense correspondences between two input images. It adapts the notion of image analogy with features extracted from a Deep Convolutional Neural Network.
Deep Image Analogy is initially described in a SIGGRAPH 2017 paper
This is an official C++ combined with CUDA implementation of Deep Image Analogy. It is worth noticing that:
- Our codes are based on Caffe.
- Our codes only have been tested on Windows 10 and Windows Server 2012 R2 with CUDA 8 or 7.5.
- Our codes only have been tested on several Nvidia GPU: Titan X, Titan Z, K40, GTX770.
- The size of input image is limited, mostly should not be large than 700x500 if you use 1.0 for parameter ratio.
© Microsoft, 2017. Licensed under an BSD 2-Clause license.
If you find Deep Image Analogy (include deep patchmatch) helpful for your research, please consider citing:
@article{liao2017visual,
title={Visual Attribute Transfer through Deep Image Analogy},
author={Liao, Jing and Yao, Yuan and Yuan, Lu and Hua, Gang and Kang, Sing Bing},
journal={arXiv preprint arXiv:1705.01088},
year={2017}
}
One major application of our code is to transfer the style from a painting to a photo.
It can also swap the styles between two artworks.
The most challenging application is converting a sketch or a painting to a photo.
It can do color transfer between two photos, such as generating time lapse.
- Windows 7/8/10
- CUDA 8 or 7.5
- Visual Studio 2013
- cuDNN
- Build Caffe at first. Just follow the tutorial here.
- Edit
deep_image_analogy.vcxproj
underwindows/deep_image_analogy
to make the CUDA version in it match yours . - Open solution
Caffe
and adddeep_image_analogy
project. - Build project
deep_image_analogy
.
You need to download models VGG-19 model before start to run a demo. Go to windows/deep_image_analogy/models/vgg19/
folder and download:
Open main.cpp
in windows/deep_image_analogy/source/
to see how to run a demo. You need to set several parameters which have been mentioned in the paper. To be more specific, you need to set
- path_model, where the VGG-19 model is.
- path_A, the input image A.
- path_BP, the input image BP.
- path_output, the output path.
- GPU Number, GPU ID you want to run this experiment.
- Ratio, the ratio to resize the inputs before sending them into the network.
- Blend Weight, the level of weights in blending process.
- Flag of WLS Filter, if you are trying to do photo style transfer, we recommend to switch this on to keep the structure of original photo.
We also provide a pre-built executable file in folder windows/deep_image_analogy/exe/
, don't hesitate to try it.
To run this deep_image_analogy.exe
, you need to write a command line as:
deep_image_analogy.exe ../models/ ../demo/content.png ../demo/style.png ../demo/output/ 0 0.5 2 0
which means
- path_model=
../models/
- path_A=
../demo/content.png
- path_BP=
../demo/style.png
- path_output=
../demo/output/
- GPU Number=
0
- Ratio=
0.5
- Blend Weight=
2
- Flag of WLS Filter=
0
(0
: WLS filter disabled,1
: WLS filter enabled, only required for the case of photo to photo)
- We often test images of size 600x400 and 448x448.
- We set ratio to 1.0 by default. Specifically, for face (portrait) cases, we find ratio = 0.5 often make the results better.
- Blend weight controls the result appearance. If you want the result to be more like original content photo, please increase it; if you want the result more faithful to the style, please reduce it.
- For the four applications, our settings are mostly (but not definitely):
- Photo to Style: blend weight=3, ratio=0.5 for face and ratio=1 for other cases.
- Style to Style: blend weight=3, ratio=1.
- Style to Photo: blend weight=2, ratio=0.5.
- Photo to Photo: blend weight=3, ratio=1.
Our codes acknowledge Eigen, PatchMatch, lbfgs and Caffe. We also acknowledge to the authors of our image and style examples but we do not own the copyrights of them.