Face cartoonization code is added. See here.
This repo heavily uses codes from [this repo]
Dataset [link]
256*256 cartoon face images collected from on-going NAVER Webtoon series.
num titles 58
total images: 17662
Training detail: batch 12, transfer from FFHQ, non-leaking augmentation
| Iteration | FID |
|---|---|
| FFHQ | 256.54 |
| 50k | 11.73 |
| 100k | 9.29 |
| 150k | 8.87 |
| 200k | 8.11 |
| 250k | 7.41 |
[paper] [model]
SwapAE is a fully unsupervised generative model that embeds images into structure and style codes (similar to MUNIT). In SwapAE, the style encoder is forced to extract the global texture of the image by explicitly matching the patch statistics of the original image and swap-generated image (patch co-occurrence loss).
Training detail: batch 10, iteration 500k, single co-occurrence patch per sample
Simply interpolating the structure code didn't work well. The regional structure code editing method introduced in the original paper might work.
Injecting target style codes from the very first layer of the decoder often changes the whole identity of the original character. Detailed structures can be better preserved by applying the source style in the first few layers.
swap location = 2
swap location = 3
For the webcomics data domain, injecting target style codes from the 2nd~3rd layer gives pleasing style-transfer results.
swap location = 7
The last few layers of the generator control the overall color.
From Slam Dunk
Finetuning the webtoon-trained SwapAE model with real human faces for small amount of steps improves the robustness. Using the StyleGAN as a style image generator, exploration of styles without real webtoon data is also possible. See train_aux.py and face2webtoon.ipynb. An unconditional lightweight model can be found here.
Without finetuning, EdgeExtraction [code] + FacialCartoonization [code] → SwapAE also produces nice results.
Failures
