squeekeek/ml

Foundations of Machine Learning Course Materials.

Foundations of Machine Learning Course Materials.

This is the repository of my course in Foundations of Machine Learning (EE 298M/CoE 197M). Similar to my previous course in Deep Learning, I would like to strike a balance between theory and practical implementation of concepts. The course materials are still under development. Please expect occasional errors, missing parts, bugs, etc. Apologies in advance.

With some modifications, the theory part is based on Mathematics for Machine Learning book. It is freely available at this link.

Thank you! I greatly appreciate the authors for having the book available online. This is helpful for students who have limited resources.

As much as possible, code examples are written to better understand key concepts. The code examples in this course are in Jupyter Notebook. I tried using Google Colab but encountered errors in saving my notebooks. So, I switched to Jupyter Notebook. In theory, you can upload the notebooks here to Google Colab.

Roadmap

0. Why Machine Learning - Importance of Foundations of Machine Learning, Course Outline
   • Lecture Notes
1. Linear Algebra - Tensors, Operations, Basis, Rank, Spaces/Subspaces, Groups, Linear Mappings
   • Lecture Notes
   • Notebook (Code Examples):
     • Introduction to Numpy
     • Linear Regression
     • Basis Vectors Transformation
2. Analytic Geometry - Distance, Metric, Norm, Inner Product, Basis, Projection, Gram-Schmidt, Rotation
   • Lecture Notes
3. Matrix Decomposition - Eigenvalues, Eigenvector, Eigendecomposition, Spectral Theorem, Singular-Value Decomposition, Matrix Approximation
   • Lecture Notes
   • Notebook (Code Examples):
     • Matrix Approximation
4. Vector Calculus - Learning, Taylor Series, Gradients, Jacobian, Backpropagation, Hessian
   • Lecture Notes
5. Probability Distributions - Distributions, Gaussian, Bayes, Sufficient Statistics, Exponential Family, Conjugacy, Transformation
   • Lecture Notes
6. Optimization - Gradient Descent, Stochastic Gradient Descent, Convex Optimization, Linear & Quadratic Programming, Convex Conjugate
   • Lecture Notes
7. Expectation Maximization (EM) and Maximum A Posteriori (MAP)
8. Regression
9. Gaussian Mixture Models
10. Support Vector Machines

Cheat Sheets

0. Numpy
1. Scikit-Learn
2. dplyr and tidyr
3. Neural Networks

Appreciation

If you find the materials in this repo useful, please give it a star or fork it.

Citation

If you find this work useful, please cite:

@misc{atienza2020ml,
  title={Foundations of Machine Learning},
  author={Atienza, Rowel},
  year={2020},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/roatienza/ml}},
}