Code written in tensorflow and numpy for various modifications of the wake-sleep algorithm. This trains a neural network with several directed layers and independent bernoulli priors on the MNIST dataset.
The command python exp_ws.py 'type' runs a variation of the wake-sleep algorithm determined by 'type' (see ws.py for the full list).
For example:
python exp_ws.py usual runs the standard wake-sleep algorithm.
python exp_ws.py mix runs a variation where the sleep sample is randomly chosen between that generated by the prior and that generated by the inference network q, with probability alpha which is annealed from 0 to 1 over the course of training.
python exp_ws.py mixy_mix is the same as above but applied to each bit of the sleep sample.
Several of the default settings can be modified in exp_ws.py.
Yura Burda and Jack Klys