Machine Learning from Scratch
This repository contains examples of popular machine learning algorithms implemented in Python with mathematics behind them being explained. Each algorithm has interactive Jupyter Notebook demo that allows you to play with training data, algorithms configurations and immediately see the results, charts and predictions right in your browser.
The purpose of this repository is not to implement machine learning algorithms by using 3rd party library one-liners but rather to practice implementing these algorithms from scratch and get better understanding of the mathematics behind each algorithm.
Table of Contents
- Machine Learning
- Deep Learning
- Optimization Algorithms
- Paper Implement
- Deep Learning with Tensorflow
Machine Learning
Supervised Learning
In supervised learning we have a set of training data as an input and a set of labels or correct answers for each training set as an output. Then we're training our model (machine learning algorithm parameters) to map the input to the output correctly (to do correct prediction). The ultimate purpose is to find such model parameters that will successfully continue correct input→output mapping (predictions) even for new input examples.
Regression
In regression problems we do real value predictions. Basically we try to draw a line/plane/n-dimensional plane along the training examples. In regression we deal with continuos as well as decreate data
Linear Regression
🤖- 📗 Math | Linear Regression - theory and links for further readings
▶️ Demo | Univariate Linear Regression
Polynomial Regression
🤖- 📗 Math | Polynomial Regression - theory and links for further readings
▶️ Demo | Univariate Polynomial Regression
🤖 Lasso/Ridge Regression
▶️ Demo | Lasso/Ridge Regression -Both regressions are explained with Neural Network. But you can apply in any algorithm also.
🤖 Support Vector Machines
SVM construct a hyper plane in high dimension which can be used for Classification , regression , or outlier detection.
- 📗 Math | SVM - theory and code
▶️ Demo | SVM Hard Margin - theory and code▶️ Demo | SVM Soft Margin - theory and code▶️ Demo | SVM Kernel Trick - theory and code▶️ Demo | SVM Sequential Minimal Optimization( SMO ) - theory and code▶️ Demo | SVM (Multi-class) - theory and code
🤖 K nearest neighbors
- KNN (K — Nearest Neighbors) is one of many (supervised learning) algorithms used in data mining and machine learning, it’s a classifier algorithm where the learning is based “how similar” is a data (a vector) from other
- 📗 Math | KNN - theory
▶️ Demo | KNN - code from scratch.
Classification
In classification problems we split input examples by certain characteristic.
_Usage examples: benign-malignent-data, wine-quality, MNIST handwritten.
🤖 Logistic Regression
- 📗 Math | Logistic Regression - theory and links for further readings
▶️ Demo | Logistic Regression (Linear Boundary)▶️ Demo | Multivariate Logistic Regression | Wine-quality▶️ Demo | Multivariate Logistic Regression | Benign-Malignent
🤖 Naive Bayes
- 📗 Math | Naive Bayes Classifier - theory and links for further readings
▶️ Demo | Bivariate Naive Bayes Classifier | Benign-Malignent
🤖 Decision Tree
-
▶️ Demo | Decision Tree | Wine-quality from scratch
Ensemble Method
Ensemble learning is a machine learning paradigm where multiple models (often called “weak learners”) are trained to solve the same problem and combined to get better results. The main hypothesis is that when weak models are correctly combined we can obtain more accurate and/or robust models.
Unsupervised Learning
Unsupervised learning is a branch of machine learning that learns from test data that has not been labeled, classified or categorized. Instead of responding to feedback, unsupervised learning identifies commonalities in the data and reacts based on the presence or absence of such commonalities in each new piece of data.
Dimentional Reduction
In dimentional reduction we select 'K' features from given 'n' features by using some techniques.
🤖 Principal Component Analysis
- 📗 Math | Principal Component Analysis - theory and explanation
▶️ Demo | Principal Component Analysis - code
🤖 Non Negative Matrix Factorization
-
📗 Theory | Non-negative Matrix Factorization - theory and explanation
-
▶️ Demo | Non-negative Matrix Factorization - OOP's implement -
▶️ Demo | Non-negative Matrix Factorization - simple implement from scratch
🤖 Singular Value Decomposition
▶️ Demo | SVD - from scratch
Clustering
Clustering is the task of dividing the population or data points into a number of groups such that data points in the same groups are more similar to other data points in the same group and dissimilar to the data points in other groups.
🤖 K-Mean
-
📗 Theory | K-Mean - theory
-
▶️ Demo | K-Mean - from scratch
🤖 DBSCAN
Deep Learning
🤖 Perceptron
Perceptron is similar to SVM it also construct a hyper plane in high dimension if data is linearly seperable.
- 📗 Theory | Perceptron - Theory of perceptron
▶️ Demo | Perceptron - code
🤖 Artificial Neural Network
- 📗 Theory | Artificial Neural Network - Theory of ANN
- 📗 Math | Regression with Artificial Neural Network - theory and code
▶️ Demo | Bivariate Classification with Artificial Neural Network▶️ Demo | Multivariate Classification with Artificial Neural Network using Sigmoid function▶️ Demo | Multivariate Classification with Artificial Neural Network using ReLu function▶️ Demo | Multivariate Classification with Artificial Neural Network ( DevnagariHandwrittenDataSet )- complete code from scratch ( Accuracy 73 % )▶️ Demo | Multivariate Classification with Artificial Neural Network ( MNIST )- complete code from scratch (Accuracy 92 %)▶️ Demo | Bunch of Activations
🤖 Convolutional Neural Network
▶️ Demo | CNN
🤖 Recurrent Neural Network
A recurrent neural network (RNN) is a class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence. This allows it to exhibit temporal dynamic behavior.
-
📗 Theory | RNN - theory and explanation
-
▶️ Demo | RNN - Vanilla RNN for Single-Batch from scratch -
▶️ Demo | RNN - Vanilla RNN for Multi-Batch from scratch -
📗 Theory | RNN - Derivation of Back Propagation through Time(BPTT).
Optimization Algorithms
🤖 Gradient Decent
🤖 Gradient Decent Check
🤖 Gradient Decent with Mini Batch
🤖 Gradient Decent with Adam Optimization
🤖 Gradient Decent with Momentum Optimization
🤖 Gradient Decent with RMSProp Optimization
🤖 Newton's Raphson Method
- 📗 Math | Newton's Raphson Method - Theory
▶️ Demo | Newton's Raphson Method - Implementation with ROC and AUC Curve
Paper Implement
Paper Implement
🤖Deep Learning with Tensorflow
Regression
▶️ Demo
Classification
▶️ Demo
Complex Modelling using Functional API
▶️ Demo
Tensorboard
▶️ Demo
Hyperparameter Fine Tuning
▶️ Demo
Tensor and Operations
▶️ Demo
Custom Model Building
▶️ Demo
Loading and Preprocessing Large Data
▶️ Demo
CNN with Tensorflow
▶️ Demo
Sequential Modelling
▶️ Demo
Character Level Modelling
▶️ Demo
Stateless RNN
▶️ Demo
Stateful RNN
▶️ Demo
Word Level Modelling
▶️ Demo
Sentiment Analysis
▶️ Demo
Encoder-Decoder
▶️ Demo
BiDirectional Layer
▶️ Demo
Beam Search
▶️ Demo
Attention
▶️ Demo
Transformers Multi Head Attention
▶️ Demo
NLP with HuggingFace and Transformers
▶️ Demo
UnderComplete Linear AutoEncoder
▶️ Demo
Stacked AutoEncoder
▶️ Demo
Convolutional AutoEncoder
▶️ Demo
Recurrent AutoEncoder
▶️ Demo
Denoising AutoEncoder
▶️ Demo
Sparse AutoEncoder
▶️ Demo
Variational AutoEncoder
▶️ Demo
Generative Adversarial Networks
▶️ Demo
Deep Convolutional GAN
▶️ Demo
Hasing using Binary AutoEncoder
▶️ Demo
Denoising AutoEncoder 3 Channel Image
▶️ Demo
Model Deployment
▶️ Demo
Topic Modelling
▶️ Demo
Prerequisites
Installing Python
Make sure that you have Python installed on your machine.
You might want to use venv standard Python library
to create virtual environments and have Python, pip
and all dependent packages to be installed and
served from the local project directory to avoid messing with system-wide packages and their
versions.
Installing Dependencies
Install all dependencies that are required for the project by running:
Datasets
The list of datasets that are being used for Jupyter Notebook demos may be found in DataSet Folder.