GANs in Computer Vision

Abstract

In this article-series we are reviewing the most fundamental works of Generative Adversarial Networks in Computer Vision. We start from the very beginning from concepts such as generative learning, adversarial learning. We provide some code and illustrations for educational purposes. The goal is to focus on the intuition of the models, by tackling the multiple problems that arise when training a GAN. We have thoroughly analyzed more than 20 papers in 6 different articles in a chronological order. We will continue to update the GAN series, based on the newer publications or older ones that we skipped. We do hope that this series will provide you a big overview of the field, so that you will not need to read all the literature by yourself, independent of your background on GANs.

Update 07/2020: free ebook is realesed in the AI summer website

Introduction to generative learning (part 1)

Link to the article: Introduction to generative learning - part 1

First Author	Published	Title	Code	Conference/Journal
Ian J. Goodfellow	10 Jun 2014	Generative Adversarial Networks	PyTorch	NIPS
Mehdi Mirza	6 Nov 2014	Conditional Generative Adversarial Nets	Tensorflow	arXiv
Alec Radford	19 Nov 2015	Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks	Tensorflow	arXiv
Xi Chen	12 Jun 2016	InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets	Tensorflow	NIPS
Tim Salimans	10 Jun 2016	Improved Techniques for Training GANs	Tensorflow	NIPS

Conditional image synthesis and 3D object generation (part2)

Link to the article: Conditional image synthesis and 3D object generation - part 2

First Author	Published	Title	Code	Conference/Journal
Augustus Odena	30 Oct 2016	Conditional Image Synthesis With Auxiliary Classifier GANs	Keras TF PyTorch	arXiv
Jiajun Wu	24 Oct 2016	Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling	PyTorch	NIPS
Zinan Lin	12 Dec 2017	PacGAN: The power of two samples in generative adversarial networks	TF	NIPS
Phillip Isola	21 Nov 2016	Image-to-Image Translation with Conditional Adversarial Networks	TF PyTorch	arXiv
Jun-Yan Zhu	30 Mar 2017	Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks	TF PyTorch	arXiv

Improved training with Wasserstein distance, game theory control and progressively growing schemes (part3)

Link to the article: Improved training with Wasserstein distance, game theory control and progressively growing schemes - part3

First Author	Published	Title	Code	Conference/Journal
Martin Arjovsky	26 Jan 2017	Wasserstein GAN	TF PyTorch	PMLR
David Berthelot	31 Mar 2017	BEGAN: Boundary Equilibrium Generative Adversarial Networks	TF PyTorch	NIPS
Tero Karras	27 Oct 2017	Progressive Growing of GANs for Improved Quality, Stability, and Variation	TF PyTorch	ICLR

2K image and video synthesis, and large-scale class-conditional image generation (part 4)

Link to the article: 2K image and video synthesis, and large-scale class-conditional image generation - part4

First Author	Published	Title	Code	Conference/Journal
Ting-Chun Wang	30 Nov 2017	High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs	PyTorch	CVPR
Ting-Chun Wang	20 Aug 2018	Video-to-Video Synthesis	PyTorch	NIPS
Andrew Brock	28 Sep 2018	Large Scale GAN Training for High Fidelity Natural Image Synthesis	TF PyTorch	ICLR

Self-supervised adversarial training and high-resolution image synthesis with style incorporation (part 5)

Link to the article: Self-supervised adversarial training and high-resolution image synthesis with style incorporation part 5

First Author	Published	Title	Code	Conference/Journal
Ting Chen	27 Nov 2018	Self-Supervised GANs via Auxiliary Rotation Loss	PyTorch	CVPR
Tero Karras	12 Dec 2018	A Style-Based Generator Architecture for Generative Adversarial Networks	TF PyTorch	CVPR

Semantic image synthesis and learning a generative model from a single image (part 6)

Link to the article: Semantic image synthesis and learning a generative model from a single image part 6

First Author	Published	Title	Code	Conference/Journal
Taesung Park	18 Mar 2019	Semantic Image Synthesis with Spatially-Adaptive Normalization	PyTorch	CVPR
Tamar Rott Shaham	2 May 2019	SinGAN: Learning a Generative Model from a Single Natural Image	PyTorch	ICCV

Other awesome GAN repositories and resources

Play with Generative Adversarial Networks (GANs) in your browser!
The GAN Zoo: A list of all named GANs
Facebook AI GAN repo: A mix of GAN implementations including progressive growing
PyTorch multiple implementations of Generative Adversarial Networks
Another PyTorch GAN library that reproduces research results for popular GANs (CVPR 2020 Workshop)
Keras implementations of Generative Adversarial Networks.
Open Questions about Generative Adversarial Networks, Augustus Odena, Distill 2019
Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
From GAN to WGAN, Lil'Log blog 2017

The article-series can be cited as:

@article{adaloglou2020gans,
    title   = "GANs in computer vision",
    author  = "Adaloglou, Nikolas and Karagiannakos, Sergios ",
    journal = "https://theaisummer.com/",
    year    = "2020",
    url     = "https://theaisummer.com/gan-computer-vision/"
  }

The-AI-Summer/GANs-in-Computer-Vision