/AnimeGAN

A Tensorflow implementation of AnimeGAN for fast photo animation ! This is the Open source of the paper 「AnimeGAN: a novel lightweight GAN for photo animation」, which uses the GAN framwork to transform real-world photos into anime images.

Primary LanguagePython

AnimeGAN

A Tensorflow implementation of AnimeGAN for fast photo animation !     日本語
The paper can be accessed here or on the website.

AnimeGANv2, the improved version of AnimeGAN.
Online access: Be grateful to @TonyLianLong for developing an online access project, you can implement photo animation through a browser without installing anything, click here to have a try.
Pytorch version: pytorch-animeGAN. Be grateful to @ptran1203.

Some suggestions:

  1. since the real photos in the training set are all landscape photos, if you want to stylize the photos with people as the main body, you may as well add at least 3000 photos of people in the training set and retrain to obtain a new model.
  2. In order to obtain a better face animation effect, when using 2 images as data pairs for training, it is suggested that the faces in the photos and the faces in the anime style data should be consistent in terms of gender as much as possible.
  3. The generated stylized images will be affected by the overall brightness and tone of the style data, so try not to select the anime images of night as the style data, and it is necessary to make an exposure compensation for the overall style data to promote the consistency of brightness and darkness of the entire style data.

News:
      AnimeGANv2 has been released and can be accessed here.

The improvement directions of AnimeGANv2 mainly include the following 4 points:

   1. Solve the problem of high-frequency artifacts in the generated image.
   2. It is easy to train and directly achieve the effects in the paper.
   3. Further reduce the number of parameters of the generator network.
   4. Use new high-quality style data, which come from BD movies as much as possible.

Requirements

  • python 3.7
  • tensorflow-gpu 1.15.0 (ubuntu, GPU 2080Ti, cuda 10.0.130, cudnn 7.6.0)
  • opencv
  • tqdm
  • numpy
  • glob
  • argparse

Usage

1. Inference

e.g. python test.py --checkpoint_dir checkpoint/generator_Hayao_weight --test_dir dataset/test/real --style_name H

2. Convert video to anime

e.g. python video2anime.py --video video/input/お花見.mp4 --checkpoint_dir ./checkpoint/generator_Hayao_weight

3. Train

1. Download vgg19 or Pretrained model

vgg19.npy

Pretrained model

2. Download dataset

Link

3. Do edge_smooth

e.g. python edge_smooth.py --dataset Hayao --img_size 256

4. Train

e.g. python train.py --dataset Hayao --epoch 101 --init_epoch 5

5. Extract the weights of the generator

e.g. python get_generator_ckpt.py --checkpoint_dir ../checkpoint/AnimeGAN_Hayao_lsgan_300_300_1_1_10 --style_name Hayao

Results

😊 pictures from the paper - AnimeGAN: a novel lightweight GAN for photo animation




😍 Photo to Hayao Style











License

This repo is made freely available to academic and non-academic entities for non-commercial purposes such as academic research, teaching, scientific publications. Permission is granted to use the AnimeGAN given that you agree to my license terms. Regarding the request for commercial use, please contact us via email to help you obtain the authorization letter.

Author

Xin Chen, Gang Liu, Jie Chen

Acknowledgment

This code is based on the CartoonGAN-Tensorflow and Anime-Sketch-Coloring-with-Swish-Gated-Residual-UNet. Thanks to the contributors of this project.