The course is devoted to modern generative models (mostly in the application to computer vision).
We will study the following types of generative models:
- autoregressive models,
- latent variable models,
- normalization flow models,
- adversarial models,
- diffusion models (sorry, next time).
Special attention is paid to the properties of various classes of generative models, their interrelationships, theoretical prerequisites and methods of quality assessment.
The aim of the course is to introduce the student to widely used advanced methods of deep learning.
The course is accompanied by practical tasks that allow you to understand the principles of the considered models.
| Lecture | Date | Description | Lecture | Video |
|---|---|---|---|---|
| 0 | September, 8 | Intro. | slides | |
| 1 | September, 8 | Logistics. Motivation. Divergence minimization framework. Autoregressive modelling. | slides | video |
| 2 | September, 15 | Autoregressive models (MADE, WaveNet, PixelCNN, PixelCNN++). Bayesian Framework. | slides | video |
| 3 | September, 22 | Latent Variable Models. Variational lower bound. EM-algorithm. Amortized inference. | slides | video |
| 4 | September, 29 | Reparametrization trick, Variational Autoencoder. Flow models definition. Forward and reverse KL divergence. | slides | video |
| 5 | October, 6 | Residual flows (Planar/Sylvester flows). Autoregressive flows (MAF/IAF/RealNVP). | slides | video |
| 6 | October, 13 | Linear flows (Glow). Posterior collapse and decoder weakening. Tigher ELBO (IWAE). | slides | video |
| 7 | October, 20 | ELBO surgery. VampPrior + flow-based VAE prior. | slides | video |
| 8 | October, 27 | Flows-based VAE posterior vs prior. Uniform and variational dequantization. Disentanglement learning (beta-VAE). | slides | video |
| 9 | Novermber, 10 | Disentanglement learning (DIP-VAE + summary). Likelihood-free learning. GAN theorem. | slides | video |
| 10 | Novermber, 17 | Vanishing gradients and Mode collapse. KL vs JSD. DCGAN. Wasserstein GAN. WGAN-GP. | slides | video |
| 11 | Novermber, 24 | Spectral Normalization GAN. f-divergence minimization. GAN evaluation (Inception score, FID). | slides | video |
| 12 | December, 1 | GAN evaluation (Precision-Recall). GAN models (Self-Attention GAN, BigGAN, PGGAN, StyleGAN). Adversarial Variational Bayes. | slides | video |
| 13 | December, 15 | Neural ODE. Continuous-in-time NF (FFJORD). Discrete VAE (Gumbel-Softmax trick, VQ-VAE, VQ-VAE-2, DALL-E). | slides | video |
| Homework | Date | Deadline | Description | Link |
|---|---|---|---|---|
| 1 | September, 15 | September, 26 | Theory: divergences + autoregressive models. Practice: MADE on 2D and MNIST. |  or  |
| 2 | September, 26 | October, 10 | Theory: log-derivative trick. Practice: PixelCNN on MNIST, VAE on 2D. | or |
| 3 | October, 10 | October, 24 | Theory: Sylvester flows. Practice: VAE on CIFAR10, RealNVP on 2D. | or |
| 4 | October, 24 | November, 14 | Theory: ELBO surgery MI. Practice: AF prior in VAE, AR Decoder. | or |
| 5 | November, 15 | November, 28 | Theory: IW dequantization, LSGAN. Practice: Standard GAN, WGAN, WGAN-GP. | or |
| 5 | November, 29 | December, 12 | Individual tasks. |
- 6 homeworks each of 13 points = 78 points
- oral cozy exam = 26 points
- maximum points: 78 + 26 = 104 points
- telegram: @roman_isachenko
- e-mail: roman.isachenko@phystech.edu