/DeepLearningAnimePapers

A list of papers and other resources on deep learning with anime style images.

MIT LicenseMIT

DeepLearningAnimePapers

A list of papers and other resources on computer vision and deep learning with anime style images. Contributions welcome!

Contents

Anime Datasets

  • AnimeDrawingsDataset: 2000 anime/manga images with 2D pose annotations [github] (Last updated June 10 2015)
  • Manga109: 109 manga volumes from the 1970s to 2010s [link]
  • Nico-Illust: 400,000 images from Niconico Seiga and Niconico Shunga with metadata [link]
  • Danbooru2017: 2.9+ million images database from Danbooru with tags [link] (Last updated March 19 2018)
  • Danbooru2018: 3.3+ million images database from Danbooru with tags [link] (Last updated April 04 2019)
  • MyAnimeList Dataset: crawled data about 14k+ anime, 300k+ users, and 80+ million animelist records [link] (Last Updated June 29 2018)

Anime Papers

Anime Colorization

  • Into the Colorful World of Webtoons: Through the Lens of Neural Networks [semanticscholar] (2017)
  • Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN [arXiv] (June 11 2017)
  • cGAN-based Manga Colorization Using a Single Training Image [arXiv] (June 21 2017)
  • Automatic Colorization of Webtoons Using Deep Convolutional Neural Networks [link] (February 2018)
  • User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks [arXiv] (August 10 2018)

Anime Face Recognition

  • A Multi-Label Convolutional Neural Network for Automatic Image Annotation [link] (July 5 2015)
  • Disentangling Style and Content in Anime Illustrations [arXiv] (May 26, 2019)

Anime Generation

Anime Image-to-Image Translation

  • Improving Shape Deformation in Unsupervised Image-to-Image Translation [arXiv] (August 13 2018)
  • Landmark Assisted CycleGAN for Cartoon Face Generation [arXiv] (July 2 2019)

Anime Inpainting

Anime Pose Estimation

  • Pose Estimation of Anime/Manga Characters: A Case for Synthetic Data [ACM] (December 4 2016)

Anime Sketch Editing

  • Sketch Simplification by Classifying Strokes [IEEE] (December 4 2016)
  • Mastering Sketching: Adversarial Augmentation for Structured Prediction [Project page with paper link] (January 18 2018) Note the version of this paper on arXiv is outdated.
  • Real-Time Data-Driven Interactive Rough Sketch Inking [Project page with paper link] (January 18 2018)

Anime Style Transfer

  • Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN [arXiv] (June 11 2017)
  • Anime Style Space Exploration Using Metric Learning and Generative Adversarial Networks [arXiv] (May 21 2018)

Anime Super Resolution

Anime Misc

  • Deep Extraction of Manga Structural Lines [ACM] (July 2017)
  • A Survey of Comics Research in Computer Science [arXiv] (April 16 2018)

Anime Non-Deep Learning

  • DrawFromDrawings: 2D Drawing Assistance via Stroke Interpolation with a Sketch Database [PubMed] (2016)
  • Sketch-based Manga Retrieval using Manga109 Dataset [SpringerLink] (November 9 2016)
  • Interactive Region Segmentation for Manga [IEEE] (December 4 2016)
  • Face Detection and Face Recognition of Cartoon Characters Using Feature Extraction [IIEEJ] (2012)

General Papers

Image Colorization

  • Real-Time User-Guided Image Colorization with Learned Deep Priors [arXiv] (May 8 2017)

Image Generation

Image Inpainting

Title Contributions Shortcomings Maximum Input Size Code?
Context Encoders: Feature Learning by Inpainting [arXiv] (April 25 2016)
  • First use of CNNs in image inpainting.
  • Utilizes an adversarial loss
  • Completed regions blurry.
 227 x 227 x 3 for random region and 128 x 128 x 3 for center region [official torch implementation]
Semantic Image Inpainting with Deep Generative Models [arXiv] (July 26 2016)
  • Missing content infered by searching for closest encoding of the corrupted image in the latent image manifold.
  • No end to end training.
  • IMHO, generating images is harder than inpainting images because with inpainting, there is always ground truth present. So converting inpainting to the harder problem of generating images might not be the way to go.
 64 x 64 x 3, arbitrary mask [tensorflow implementation]
Globally and Locally Consistent Image Completion [link] (2017)
  • Dilated convolutions
  • 2 discriminators: one local discriminator for the completed region and one global discriminator for whole image
  • Long training time.
  • Poisson blending needed.
  • Complex training process. Completion network is trained, then the completion network is fixed and discriminators are trained, then finally both are trained.
 256 x 256 x 3 [official torch implementation. no training code] [tensorflow implementation. missing gan loss] [tensorflow implementation. has UI]
Image Inpainting using Multi-Scale Feature Image Translation [arXiv] (November 23 2017)
  • 2 stages: coarse prediction and refinement through feature based texture swapping
  • Framework can be adapted to multi-scale
 256 x 256 x 3 input, arbitrary mask Soon
Context-Aware Semantic Inpainting [arXiv] (December 21 2017)
Light-weight pixel context encoders for image inpainting [arXiv] (January 17 2018)
High Resolution Face Completion with Multiple Controllable Attributes via Fully End-to-End Progressive Generative Adversarial Networks [arXiv] (January 23 2018)
  • Conditioned on facial attributes.
  • Progressive growing of GANs.
  • Three new losses: attribute, feature, and boundary.
  • Fails to learn low level skin features.
  • Long training time.
 1024 x 1024 x 3, arbitrary mask Soon
Deep Structured Energy-Based Image Inpainting [arXiv] (January 24 2018)
Image Inpainting for Irregular Holes Using Partial Convolutions [arXiv] (April 20 2018)
  • Introduces partial convolutions, which exclude information from the mask.
  • No post processing.
 512 x 512 x 3, arbitrary mask
Free-Form Image Inpainting with Gated Convolution [arXiv] (June 10 2018)
  • Utilizes gated convolutions.
  • State of the art inpainting for irregular masks.
  • No post processing.
 512 x 512 x 3, arbitrary mask Soon

Other Repositories

Anime Repositories

  • none

General Repositories