wbj0110/ml-from-scratch

All the ML algorithms, ML models are coded from scratch by pure Python/Numpy with the Math under the hood. It works well on CPU.

Machine Learning from scratch

About

This ML repository is all about coding Machine Learning algorithms from scratch by Numpy with the math under the hood without Auto-Differentiation frameworks like Tensorflow, Pytorch, etc. Some advanced models in Computer Vision, NLP require Tensorflow to quickly get the idea written in paper.

Repository structure

As a software engineer, I follow the principle of OOP to construct the repository. You can see that NeuralNetwork class will use FCLayer, BatchNormLayer, ActivationLayer class and CNN class will use ConvLayer, PoolingLayer, FCLayer, ActivationLayer,... This helps me easily reuse every piece of code I wrote as well as for readable code.

Dependencies:

• tqdm, numpy, sklearn, matplotlib

Table of contents

• Machine Learning models:

  • Linear Regression
  • Logistic Regression
  • Softmax Regression
  • Neural Network
  • Convolutional Neural Network
  • Generative Adversarial Network
  • Support Vector Machine

• Deep Learning layers:

  • Fully-Connected Layer
  • Convolutional Layer
  • Pooling Layer
  • Activation Layer
  • BatchNorm Layer
  • Dropout Layer

• Optimization algorithms:

  • SGD
  • SGD with Momentum
  • RMSProp
  • Adam

• Weights initialization:

  • He initialization
  • Xavier/Glorot initialization

• Advanced models:

  • Attention Mechanism (Bahdanau and Luong Attention)
  • Transformer