gauravreddy08/everything-from-scratch

Implement everything in ML from scratch

Index

Machine Learning Models

Implementation	Code	Extra Notes
Linear Regression	LinearRegression.py
Logistic Regression (Binary Classification)	LogisticRegression.py
Multinomial Log Regression (Multiclass Classification)	MultinomialLogisticRegression.py
K-Nearest Neighbors (KNN)	KNN.ipynb
K-Means Clustering	KMeans.ipynb

Neural Networks

Implementation	Code	Extra Notes
MLP (Neural Network from Scratch)	MLP.py	backprop
LSTM	LSTM.py	Notes

Attention Mechanisms

Implementation	Code	Extra Notes
Self Attention	SelfAttention.py
Multi-Head Attention	MultiHeadAttention.py
Naive Multi-Head Attention	NaiveMultiHeadAttention.py
Multi-Query Attention	MultiQueryAttention.py
Grouped-Query Attention	GroupedQueryAttention.py

Normalization Techniques

Implementation	Code	Extra Notes
Batch Normalization	BatchNorm.py
Layer Normalization	LayerNorm.py
RMS Normalization	RMSNorm.py

Language Models

Implementation	Code	Extra Notes
GPT-2 Implementation	main.py
Llama 2 Implementation	main.py
Llama 3 Implementation	main.py
Llama 3.1 Implementation	main.py

More implementations coming soon...

• Qwen
• KV Cache
• Knowledge Distillation