AnimeME: Apply anime styles to your photos using GANs

Pytorch implementation of AnimeGAN for fast photo animation.

AnimeGAN: a novel lightweight GAN for photo animation - Semantic scholar

Deployments

Gradio App is deployed on Huggingface-Spaces 🤗
Backend developed using FastAPI is deployed on lightning-cloud

Documentation

Folder Structure

src: Training Code
gradio_app: code for the app deployed on huggingface-spaces
lightning_backend: code for the lightning app backend deployed on lightning-cloud.

1. Install required dependencies

Create virtual environment in python.
Install dependencies using the following command

pip install -r requirements.txt

2. Prepare dataset

Download the dataset from AnimeGANV2 repo
Extract the dataset to the desired folder. Run the below command if there are no smooth images in your style folder i.e Hayao, Paprika, etc

python src/data_process --data_dir <Path to the stored location of dataset> --dataset <The name of the style data i.e Hayao/Paprika/Shenkai> --image-size 256

3. Train animeGAN

To train the animeGAN from command line, you can run train_model.py as the following:
You can play around the parameters in params.yaml file avaiable in Params folder.

python src/train_model.py

3. Transform images

To convert images in a folder or single image, run inference_image.py, for example:

--src and --dest can be a path to file : "./images/fake.png" or something like this.

python src/infer.py --checkpoint_path <path to saved checkpoint> --source_path <path to src image> --dest_path <path to where image should be saved>

4. Run lightning app on local system

To run lightning app on your local system follow the below instructions

Navigate to lightning_backend folder and install the requirements.
Run the below command to run the app.

lightning run app app.py

This will open up for the app.
If you want to use the backend which I have deployed use the below command to curl for results

curl -X 'POST' \
  'https://mhwgp-01gjvacrnf5yhhtdeebd0bhw3t.litng-ai-03.litng.ai/api/predict' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -F 'data=<Path to your file>'

Note: I cannot guarantee how much time this API will be hosted as it may cost me later down the line.

Anime transformation results

Input	Output(Hayao style)

Training Curves

Below is how my loss looked like for overall training. All the tracking is done using ML-Flow.

Note : Batch size will has significant effect on training and results. Atleasts that's what I observed.

Things to remember

You can train this on any dataset but images should be high resolution especially the style images.
Read the paper to understand the implementation and process.
It is very important to track your experiments, I used mlflow to track my experiment and parameters.

Work in Progress

TensorRT/Onnx models.