elaysason/Generative-Learning-With-Discrete-and-Continuous-VAE

Using VAE based netural network in order to learn the disiturbtion space which the photos are getting sampled and generate new photos.

Generative-Learning-With-Discrete-and-Continuous-VAE

Using VAE based netural network in order to learn some discrete and continuous latent state representation of that data and generate new photos.

1. General
   • Background
2. Program Structure
   • Network Structure
   • Visualtions
3. Installation

General

The goal is to bulid deep learning based generative model. In order to that the network learn the disiturbtion space of the photos and sample from it.

Background

VAE is an autoencoder which provides a probabilistic manner for describing an observation in latent space. Thus, rather than building an encoder which outputs a single value to describe each latent state attribute, it will formulate encoder to describe a probability distribution for each latent attribute.

Program Structure

• models.py - Creation of the VAE netowrk.
• main.py - loads a existing model or create one and plot visualizations of it
• visualizations.py - Responible for the visualizations of the model.
• training.py - trains the model

Network-Structure

The VAE network is described as follows:

The dims of the convolution are 3-32-64, padding is 1 and stride is 2 and every convolution layer also includes Leaky relu function in both encoder and decoder.

The linear layers dims encoder:

• The linear layer dims are 64 * 4 * 4 to hidden dim.
• The linear discrete layers dims are hidden dim to discrete dim.
• The linear continuous layers dims are hidden dim to continuous dim.

The linear layers dims decoder:

• The linear layers dims are latent dim to hidden dim and hidden dim to 64 * 4 * 4 .
• The linear layers also include leaky relu function.
• We used reparameterization trick for continuous variables and Gumbel SoftMax for the discrete variables.

Visualtions

Having a look about the Visualtions of the latent spaces:

• Gender:

  

• Hair Style:

  

We can see that there is a speration but not a perfect one and in the hair which indcluds large number of categoires the model strugles with black hair for example.

Installation

1. Open the terminal

2. Clone the project by:

    $ git clone https://github.com/elaysason/Generative-Learning-With-Discrete-and-Continuous-VAE.git

3. Run the main.py file by:

    $ python main.py