/Advanced-Deep-Learning-with-Keras

Advanced Deep Learning with Keras, published by Packt

Primary LanguagePythonMIT LicenseMIT

Advanced Deep Learning with Keras

This is the code repository for Advanced Deep Learning with Keras, published by Packt. It contains all the supporting project files necessary to work through the book from start to finish.

About the Book

This book covers advanced deep learning techniques to create successful AI. Using MLPs, CNNs, and RNNs as building blocks to more advanced techniques, you’ll study deep neural network architectures, Autoencoders, Generative Adversarial Networks (GANs), Variational AutoEncoders (VAEs), and Deep Reinforcement Learning (DRL) critical to many cutting-edge AI results.

Instructions and Navigation

All of the code is organized into folders. Each folder starts with a number followed by the application name. For example, chapter2-deep-networks.

The code will look like the following:

def encoder_layer(inputs,
                  filters=16,
                  kernel_size=3,
                  strides=2,
                  activation='relu',
                  instance_norm=True):
"""Builds a generic encoder layer made of Conv2D-IN-LeakyReLU 
IN is optional, LeakyReLU may be replaced by ReLU
"""
conv = Conv2D(filters=filters,
              kernel_size=kernel_size,
              strides=strides,
              padding='same')

Related Products

Advanced Deep Learning with Keras sample code used in the book

  1. MLP on MNIST
  2. CNN on MNIST
  3. RNN on MNIST
  1. Functional API on MNIST
  2. Y-Network on MNIST
  3. ResNet v1 and v2 on CIFAR10
  4. DenseNet on CIFAR10
  1. Denoising AutoEncoders

Sample outputs for random digits:

Random Digits

  1. Colorization AutoEncoder

Sample outputs for random cifar10 images:

Colorized Images

  1. Deep Convolutional GAN (DCGAN)

Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015).

Sample outputs for random digits:

Random Digits

  1. Conditional (GAN)

Mirza, Mehdi, and Simon Osindero. "Conditional generative adversarial nets." arXiv preprint arXiv:1411.1784 (2014).

Sample outputs for digits 0 to 9:

Zero to Nine

  1. Wasserstein GAN (WGAN)

Arjovsky, Martin, Soumith Chintala, and Léon Bottou. "Wasserstein GAN." arXiv preprint arXiv:1701.07875 (2017).

Sample outputs for random digits:

Random Digits

  1. Least Squares GAN (LSGAN)

Mao, Xudong, et al. "Least squares generative adversarial networks." 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017.

Sample outputs for random digits:

Random Digits

  1. Auxiliary Classfier GAN (ACGAN)

Odena, Augustus, Christopher Olah, and Jonathon Shlens. "Conditional image synthesis with auxiliary classifier GANs. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, PMLR 70, 2017."

Sample outputs for digits 0 to 9:

Zero to Nine

  1. Information Maximizing GAN (InfoGAN)

Chen, Xi, et al. "Infogan: Interpretable representation learning by information maximizing generative adversarial nets." Advances in Neural Information Processing Systems. 2016.

Sample outputs for digits 0 to 9:

Zero to Nine

  1. Stacked GAN

Huang, Xun, et al. "Stacked generative adversarial networks." IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Vol. 2. 2017

Sample outputs for digits 0 to 9:

Zero to Nine

  1. CycleGAN

Zhu, Jun-Yan, et al. "Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks." 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017.

Sample outputs for random cifar10 images:

Colorized Images

Sample outputs for MNIST to SVHN:

MNIST2SVHN

  1. VAE MLP MNIST
  2. VAE CNN MNIST
  3. Conditional VAE and Beta VAE

Kingma, Diederik P., and Max Welling. "Auto-encoding Variational Bayes." arXiv preprint arXiv:1312.6114 (2013).

Sohn, Kihyuk, Honglak Lee, and Xinchen Yan. "Learning structured output representation using deep conditional generative models." Advances in Neural Information Processing Systems. 2015.

I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner. β-VAE: Learning basic visual concepts with a constrained variational framework. ICLR, 2017.

Generated MNIST by navigating the latent space:

MNIST

  1. Q-Learning
  2. Q-Learning on Frozen Lake Environment
  3. DQN and DDQN on Cartpole Environment

Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." Nature 518.7540 (2015): 529

DQN on Cartpole Environment:

Cartpole

  1. REINFORCE, REINFORCE with Baseline, Actor-Critic, A2C

Sutton and Barto, Reinforcement Learning: An Introduction

Mnih, Volodymyr, et al. "Asynchronous methods for deep reinforcement learning." International conference on machine learning. 2016.

Policy Gradient on MountainCar Continuous Environment:

Car

Citation

If you find this work useful, please cite:

@book{atienza2018advanced,
  title={Advanced Deep Learning with Keras},
  author={Atienza, Rowel},
  year={2018},
  publisher={Packt Publishing Ltd}
}