/PMC-Video-Animation

Primary LanguagePython

Unsupervised Coherent Video Cartoonization with Perceptual Motion Consistency

Unsupervised Coherent Video Cartoonization with Perceptual Motion Consistency. AAAI 2022. Arxiv

Paper

Setup Environment

conda env create -f environment.yaml
conda activate video-animation

Training

  1. Preparing Training Data Download datasets from this drive and unzip to datasets folder.
  2. Download pretrained vgg from here and unzip, put it to models/vgg19.npy
  3. Start training.
CUDA_VISIBLE_DEVICES=0 python train.py --exp_name with-pmc --temporal_weight 1.0

Testing

Download Pretrained Network from google drive.

Translating Images

  • Translate images in input directory and save into output directory.
python inference.py --input_path ${your_input_folder} --output_path ${your_output_folder} --model_path pretrained.ckpt

Translating Video

python translate_video.py --input_video ${your_input_video} --output_dir ${your_output_folder} --model_path pretrained.ckpt

Results

Image Results

Images
Results
Results
Results
Results

Video Results

Video Results

Visualization of Spatio-Temporal Coorelative Maps

Spatio-Temporal Coorelative Maps

Citation

@inproceedings{Liu2022UnsupervisedCV,
  title={Unsupervised Coherent Video Cartoonization with Perceptual Motion Consistency},
  author={Zhenhuan Liu and Liang Li and Huajie Jiang and Xin Jin and Dandan Tu and Shuhui Wang and Zhengjun Zha},
  booktitle={AAAI},
  year={2022}
}

Acknowledgment