/Fundamentals

During my research, I happen to enjoy implementing algorithms from scratch. Through Fundamentals, I like to share my experiences with others.

Primary LanguageJupyter Notebook

Fundamentals

With this collection of notebooks, I would like to share my experience of learning and researching machine learning techniques. To this end, on each notebook, the reader is firstly provided with a brief mathematical background knowledge. Therafter, I provided a naive Python implementation of the respective technique. During the implementation, I focused on the readability.

Recreating techniques via implementing them from scratch necessitates understanding atomic details of the respective techniques. Listen Yann LeCun, Yoshua Bengio and Andrej Karpathy. They can not be all wrong, can they ? :)

Feel free to use/modify any code provided in this repository. However, I have a request from you. Please do not forget that “No one has ever become poor by giving.” – Anne Frank. Please make a small donation to World Food Programme.

Content

Prerequisite: Linear Algebra

  1. Machine Learning
    1. Naive Bayes
    2. Regression
    3. Maximum Likelihood vs Maximum A Posteriori Estimation
    4. Support Vector Machines
    5. Loss Funciton Langscape
    6. Kernalization
    7. Generative vs Discriminator
    8. Gaussian Process
    9. Bayesian Optimization
    10. Bagging
    11. From Decision Tree to Random Forest
    12. Boosting
  2. Deep Learning
    1. Linear Classification
    2. Vanilla Nets
    3. Convolutions
    4. Forward and Backward Passes in Nets
    5. Optimization as API
    6. Vanishing Gradient
    7. Batch Normalization
    8. Dropout
    9. Recurrent Nets
    10. LSTM
    11. Generative Adversarial Network
    12. Graph Convolutional Networks
    13. Laplace Redux
  3. Reinforcement Learning
    1. Search
    2. MDP
    3. RL
    4. Deep Q-Network
  4. Numerical Optimization
    1. Descent them all:
      1. Gradient Descent
      2. Stochastic Gradient Descent
      3. Momentum
      4. Nesterov's Momentum