Wenlin-Chen/NumLayers

A Deep Learning Library Written in NumPy

NumLayers

A Deep Learning Library Written in NumPy (from scratch)

• A personal project for fun and for the purpose of learning
  
• Author: Wenlin Chen
  

Requirements

• Python 3.x
  
• NumPy
  
• Matplotlib
  
  

Getting Started

• MNIST dataset
  Download http://deeplearning.net/data/mnist/mnist.pkl.gz
  and put it in ./data/mnist/
  

• CIFAR10 dataset
  Dwonload https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz
  extract it and put the folder cifar-10-batches-py in ./data/cifar10/
  

• Cython acceleration for Conv2d and MaxPool2d (Optional, im2col_cython.pyx from Stanford CS231n)
  Run the following command from the ./utils/im2col/ directory:
  python setup.py build_ext --inplace
  

• Run the example models in ./models/
  Image Recognition by MLP and CNN
  Image Generation by GAN
  
  

Features

Layers

• Linear
  
• Dropout
  
• Convolution 2D
  
• Max Pooling 2D
  
• Batch Normalization 1D & 2D
  

Losses

• Hinge Loss
  
• Cross Entropy Loss
  
• Mean Square Error Loss
  
• Binary Cross Entropy Loss
  

Activation Functions

• Tanh
  
• ReLU
  
• Sigmoid
  
• Leaky ReLU
  

Optimizers

• Adam
  
• RMSprop
  
• Momentum and Nesterov
  
• Stochastic Gradient Descent
  

Training Tools

• L2 Regularization
  
• Data Augmentation
  
• Learning Rate Decay
  
  

Example Models

• ./models/mlp_mnist.py
  
  
  
• ./models/mlp_cifar10.py
  
  
  
• ./models/cnn_mnist.py
  
  
  
• ./models/cnn_cifar10.py