weiliangxiao/mxnet-the-straight-dope

Much easy, so MXNet. Wow.

mxnet-the-straight-dope

Much easy, so MXNet. Wow.

Abstract

MXNet is widely used in production environments owing to its strong reputation for speed. Now with gluon, MXNet's new imperative interface (alpha), doing research in MXNet is easy.

This repo contains an incremental sequence of tutorials to make learning gluon easy (meta-easy?). These tutorials are designed with public presentations in mind and are composed as Jupyter notebooks where each notebook corresponds to roughly 20 minutes of rambling and each codeblock could correspond to roughly one slide.

Inspiration

In creating these tutorials, I've borrowed heavily from some of the resources that were most useful when I learned how to program with Theano and PyTorch, specifically:

• Soumith Chintala's helladope "Deep Learning with PyTorch: A 60 Minute Blitz"
• Alec Radford's bare-bones intro to Theano
• Video of Alec's awesome intro to deep learning with Theano

Preliminaries

To run these notebooks, you'll want to build mxnet from source. Fortunately, this is easy (especially on Linux) if you follow these instructions. You'll also want to install Jupyter and use Python 3 (because it's 2017).

Table of contents

• 1 - Manipulating data with NDArray
• 2 - Automatic differentiation via autograd
• 3a - Linear Regression (from scratch!)
• 3b - Linear Regression (with gluon!)
• 4a - Multiclass Logistic Regression (from scratch!)
• 4b - Multiclass Logistic Regression (with gluon!)
• 5a - Multilayer Perceptrons (from scratch!)
• 5b - Multilayer Perceptrons (with gluon!)
• 6a - Convolutional Neural Network (from scratch!)
• 6b - Convolutional Neural Network (with gluon!)

Choose your own adventure

I've designed these tutorials so that you can traverse the curriculum in one of three ways.

• Anarchist - Choose whatever you want to read, whenever you want to read it.
• Imperialist - Proceed throught the tutorials in order (1, 2, 3a, 3b, 4a, 4b, ...). In this fashion you will be exposed to each model first from scratch, writing all the code ourselves but for the basic linear algebra primitives and automatic differentiation.
• Capitalist - If you would like to specialize to either the raw interface or the high-level gluon front end choose either (1, 2, 3a, 4a, ...) or (1, 2, 3b, 4b, ...) respectively.

Roadmap

• GANs (DCGAN, InfoGAN, EBGAN, Wasserstein GAN, SD-GAN)
• Simple RNN (from scratch and w gluon)
• LSTM (from scratch and w gluon)
• GRU
• DQN
• Sequence to Sequence
• Sequence to Sequence with Attention
• Weight uncertainty Bayes-by-Backprop neural networks
• Residual networks
• Latent factor models
• Word2Vec