Anime2Sketch

Anime2Sketch: A sketch extractor for illustration, anime art, manga

Introduction

The repository contains the testing codes and pretrained weights for Anime2Sketch. Slightly modified by bycloud for Windows installation.

Anime2Sketch is a sketch extractor that works well on illustration, anime art, and manga. It is an application based on the paper "Adversarial Open Domain Adaption for Sketch-to-Photo Synthesis".

Requirements

Anaconda3 Prompt is used

You can get it here

Or you can run it from Google Colab

Get Started

Installation

conda create -n a2s python=3.6

conda activate a2s

conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=10.2 -c pytorch

or if you are using NVIDIA 30 series

conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorch

Download Pretrained Weights

Please download the weights from GoogleDrive, and put it into the weights/ folder.

Test

python test.py --dataroot /your_input/dir --load_size 512 --output_dir /your_output/dir

The above command includes three arguments:

dataroot: your test file or directory
load_size: due to the memory limit, we need to resize the input image before processing. By default, we resize it to 512x512.
output_dir: path of the output directory

Run our example on a specific image:

python test.py --dataroot test_samples/madoka.jpg --load_size 512 --output_dir results/

or running on a folder:

python test.py --dataroot test_samples/*FOLDER_NAME* --load_size 512 --output_dir results/

or running on a video:

Get ffmpeg

conda install -c conda-forge ffmpeg

We would need to extract all the frames first. Find out the FPS of the video by right clicking it. Drag the video into test_samples folder, and create the folder based on the mp4's name for ease use later.

ffmpeg -i test_samples/*YOUR_MP4_NAME*.mp4 -vf fps=*YOUR_FPS_COUNT* test_samples/*YOUR_MP4_NAME*/%06d.jpg

Run the main module:

python test.py --dataroot test_samples/*FOLDER_NAME* --load_size 512 --output_dir results/*FOLDER_NAME*

Put the images back together:

ffmpeg -i results/*YOUR_MP4_NAME*/%06d.jpg -vf fps=*YOUR_FPS_COUNT* results/*YOUR_MP4_NAME*.mp4

Train

Check the main repository for more info

More Results

Our model works well on illustration arts: Turn handrawn photos to clean linearts: Simplify freehand sketches: And more anime results:

License

This project is released under the MIT License.

Citations

@misc{Anime2Sketch,
  author = {Xiaoyu Xiang, Ding Liu, Xiao Yang, Yiheng Zhu, Xiaohui Shen},
  title = {Anime2Sketch: A Sketch Extractor for Anime Arts with Deep Networks},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/Mukosame/Anime2Sketch}}
}

@misc{xiang2021adversarial,
      title={Adversarial Open Domain Adaption for Sketch-to-Photo Synthesis}, 
      author={Xiang, Xiaoyu and Liu, Ding and Yang, Xiao and Zhu, Yiheng and Shen, Xiaohui and Allebach, Jan P},
      year={2021},
      eprint={2104.05703},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

bycloudai/Anime2Sketch-Windows