dali1981/mathematical-methods-in-deep-learning-ipython

Python code (in the form of Jupyter ipython notebooks) to support the book "Math and Architectures of Deep Learning" (Krishnendu Chaudhury with Ananya Ashok, Sujay Narumanchi, Devashish Shankar).

Math and Architectures of Deep Learning

Python code (in the form of Jupyter ipython notebooks) to support the book "Math and Architectures of Deep Learning".

Code contributors: Ananya Ashok, Sujay Narumanchi, Devashish Shankar, Krishnendu Chaudhury.

This repository contains the example code - mostly in Numpy and PyTorch - corresponding to the theoretical topics introduced in the book. The code listings are organized in chapters that correspond to the main book.

Installation

1. Clone the repository: git clone https://github.com/krishnonwork/mathematical-methods-in-deep-learning-ipython.git
2. Create virtual environment: virtualenv venv --python=python3 (you may need to do pip install virtualenv first)
3. Activate virtual environment: source venv/bin/activate
4. Change directory: cd mathematical-methods-in-deep-learning-ipython
5. Install dependencies: pip install -r requirements.txt
6. Navigate to the python directory: cd python
7. Start jupyter: jupyter notebook

This will redirect you to a browser window with the ipython notebooks

Note: Ensure to use Python3 to run the notebooks

Table of Contents

• Chapter 2:

  • 2.2 Intro to Vectors
  • 2.4 Intro to Matrices, Tensors and Images
  • 2.7 Basic Vector and Matrix operations
  • 2.12.5 Solving an overdetermined system using pseudo inverse
  • 2.13 Eigenvalues and Eigenvectors
  • 2.14 Rotation Matrices
  • 2.15 Matrix Diagonalization
  • 2.16 Spectral Decomposition of a Symmetric Matrix
  • 2.17 Finding the axes of a hyper-ellipse

• Chapter 3

  • 3.4 Common code for chapter 3
  • 3.4.1 Gradient Descent
  • 3.4.2 Non-linear Models
  • 3.4.3 A Linear Model for the cat-brain

• Chapter 4

  • 4.3.2 Common code for chapter 4
  • 4.3.2 PCA on synthetic correlated data
  • 4.3.2 PCA on synthetic uncorrelated data
  • 4.3.3 PCA on synthetic correlated non-linear data
  • 4.4.4 Linear system solving via SVD
  • 4.4.5 PCA computation via SVD
  • 4.5.3 LSA on a toy dataset
  • 4.5.4 LSA/SVD on a 500 × 3 dataset

• Chapter 5

  • 5.9.1 Uniform Random Distribution
  • 5.9.2 Normal Distribution
  • 5.9.3 Binomial Distribution
  • 5.9.4 Multinomial Distribution
  • 5.9.5 Bernoulli Distribution

• Chapter 6

  • 6.2.2 Entropy
  • 6.3.1 Cross Entropy
  • 6.4.2 KL Divergence
  • 6.7.1 Model Parameter Estimation
  • 6.8.2 Gaussian Mixture Modelling

• Chapter 7

  • 7.2.3 Perceptron
  • 7.2.4 Modeling logic gates with perceptrons
  • 7.4.3 Approximating surfaces with perceptrons

• Chapter 8

  • 8.4 Forward and Backward pass
  • 8.5 Training a neural network

• Chapter 9

  • 9.1 Loss Functions
  • 9.2.3 Stochastic Gradient Descent
  • 9.2.4 Momentum