moshesipper/Applied-Machine-Learning-Course

This course covers the applied side of algorithmics in machine learning, with some deep learning and evolutionary algorithms thrown in as well.

Applied Machine Learning

This course covers the applied side of algorithmics in machine learning, with some deep learning and evolutionary algorithms thrown in as well.

Prerequisites: Design of Algorithms, Algebra 2, Calculus 2, Probability and Statistics

Moshe Sipper’s Cat-a-log of Writings

Some Pros and Cons of Basic ML Algorithms, in 2 Minutes

Additional Resources (Cheat Sheets, Vids, Reads, Books, Software, Datasets)

---

Syllabus

❖ Math ❖ Python ❖ Artificial Intelligence ❖ Date Science ❖ Machine Learning Intro ❖ Scikit-learn ❖ ML Models ❖ Decision Trees ❖ Random Forest ❖ Linear Regression ❖ Logistic Regression ❖ Linear Models ❖ Regularization: Ridge & Lasso ❖ AdaBoost ❖ Gradient Boosting ❖ AddGBoost ❖ Ensembles ❖ XGBoost ❖ Comparing ML algorithms ❖ Gradient Descent ❖ SVM ❖ Bayesian ❖ Metrics ❖ Data Leakage ❖ Dimensionality Reduction ❖ Clustering ❖ Hyperparameters ❖ Some Topics in Probability ❖ Feature Importances ❖ Semi-Supervised Learning ❖ Neural Networks ❖ Deep Learning ❖ DL and AI ❖ Evolutionary Algorithms: Basics ❖ Evolutionary Algorithms: Advanced ❖ Large Language Models

---

Topics (according to order of instruction)

(: my colab notebooks, : my medium articles)

• Math

  • Mathematics for ML: ppt, pdf, alternate

• Python

  • Python Tutorial With Google Colab
  • Learn Python Programming
  • Python
  • PyCharm
  • Pandas
  • NumPy
  • NumPy
  • Numba
  • np.dot vs loop example

• Artificial Intelligence

  • Computing Machinery and Intelligence
  • Four Takeaways on the Race to Amass Data for A.I.

• Date Science

  • Data Science Landscape
  • Building the Machine Learning Model
  • 21 Most Important (and Must-know) Mathematical Equations in Data Science
  • spurious correlations

• Machine Learning Intro

  • Machine Learning: history, applications, recent successes
  • How to avoid machine learning pitfalls
  • Top 10 machine learning algorithms with their use-cases
  • Introduction to machine learning
  • Train/Val/Test
  • Simple weather example
  • Iris knn (map)
  • kfold
  • Overfitting

• Scikit-learn

  • Machine learning with scikit-learn
  • A Minimal Example of Machine Learning (with scikit-learn)
  • 19 Most Elegant Sklearn Tricks I Found After 3 Years of Use
  • Toy datasets (sklearn)

• ML Models

  • Machine learning models

• Decision Trees

  • Decision trees
  • Decision trees

• Random Forest

  • Random Forests
  • Random Forests

• Linear Regression

  • Linear Regression
  • LinearReg
  • A Quick Summary of Linear Regression

• Logistic Regression

  • Logistic Regression
  • Logistic Regression
  • Cross-Entropy Loss
  • Linear Regression vs Logistic Regression in a Nutshell
  • LinVsLog
  • PolynomialFeatures

• Linear Models

  • Optimization of linear models

• Regularization: Ridge & Lasso

  • Ridge vs. Lasso
  • notes on regularization in ML

• AdaBoost

  • Adaptive Boosting

• Gradient Boosting

  • Gradient Boosting
  • Why tree gradients give you a boost

• AddGBoost

  • AddGBoost
  • Strong(er) Gradient Boosting

• Ensembles

  • Two’s Company, Three’s an Ensemble

• XGBoost

  • XGBoost

• Comparing ML algorithms

  • Comparing supervised learning algorithms
  • How to find the best performing Machine Learning algorithm
  • load_boston removal
  • Choosing the right estimator
  • Questionable practices in machine learning

• Gradient Descent

  • Gradient Descent
  • Least Squares
  • Least Squares
  • Stochastic Gradient Descent
  • Stochastic Gradient Descent Algorithm With Python and NumPy
  • Gradient Descent With Momentum

• SVM

  • Support Vector Machine
  • Plot different SVM classifiers in the iris dataset
  • SVM’s Kernel Trick in a minute

• Bayesian

  • Multinomial Naive Bayes

• Metrics

  • sklearn.metrics
  • Confusion Matrix
  • Confusion Matrix & Metrics in a minute
  • Sensitivity and Specificity
  • Sensitivity and specificity
  • ROC and AUC
  • balanced accuracy
  • balanced accuracy
  • various metrics from CM

• Data Leakage

  • Data Leakage Basics, with Examples in Scikit-Learn
  • Could machine learning fuel a reproducibility crisis in science?

• Dimensionality Reduction

  • 11 Different Uses of Dimensionality Reduction
  • PCA
  • PCA
  • PCA vs LR
  • pca
  • t-SNE
  • tsne

• Clustering

  • Clustering
  • Hierarchical clustering
  • K-means clustering
  • kmeans
  • From Data to Clusters: When is Your Clustering Good Enough?

• Hyperparameters

  • Model Parameter vs. Hyperparameter
  • Hyperparameter tuning
  • Hyperparameter tuning
  • Optuna
  • optuna
  • Evaluating Hyperparameters in Machine Learning

• Some Topics in Probability

  • p-values
  • how to calculate p-values
  • p-hacking
  • Probability is not Likelihood
  • t-test
  • t-test
  • t-test vs p-value
  • scpipy ttest
  • chi-square
  • 17 Statistical Hypothesis Tests in Python
  • permutation test (AddGBoost + code)
  • The Permutation Test: A Data Scientist’s BFF

• Feature Importances

  • Decision Tree
  • Feature importances with a forest of trees
  • Explainable AI explained! | #4 SHAP
  • SHAP Values Explained

• Semi-Supervised Learning

  • Boosting vs. semi-supervised learning
  • LabelPropagation
  • Semi-Supervised Learning: How To Overcome the Lack of Labels

• Neural Networks

  • Neural networks
  • Neural networks
  • Tinker With a Neural Network in Your Browser

• Deep Learning

  • Neural Networks with À La Carte Selection of Activation Functions
  • PyTorch
  • PyTorch
  • Double Descent
  • Overparameterization, Backpropagation, Alimentation: Them and Us
  • No, Kernels & Filters Are Not The Same
  • conv demo
  • convolution
  • A simple image convolution
  • Implementing Image Processing Kernels from scratch using Convolution in Python
  • Introduction to image generation (diffusion)
  • Loss is Boss and other articles in the DL section
  • Neural Networks from Scratch

• DL and AI

  • Growth of AI computing
  • AI move from Academia
  • Artificial General Intelligence Is Not as Imminent as You Might Think

• Evolutionary Algorithms: Basics

  • Evolutionary Computation
  • How to Build a Genetic Algorithm from Scratch in Python with Just 33 Lines of Code
  • Evolutionary Algorithms, Genetic Programming, and Learning
  • Tiny GA
  • EC-KitY
  • Genetic Programming slides
  • Tiny GP
  • GP Tutorial
  • GP vids
  • GP slides
  • Natural Computing

• Evolutionary Algorithms: Advanced

  • Multi-Objective Optimization
  • Schema theorem
  • Linear GP
  • Cartesian GP
  • Grammatical Evolution
  • Coevolutionary Computation
  • New Pathways in Coevolutionary Computation
  • Novelty search
  • Humies
  • Evolutionary Art
  • Building Activation Functions for Deep Networks
  • Evolutionary Adversarial Attacks on Deep Networks

• Large Language Models

  • Introduction to large language models
  • A Tiny Large Language Model (LLM), Coded, and Hallucinating
  • Large Language Models from scratch
  • Large Language Models: Part 2
  • Scikit-LLM
  • Coding a ChatGPT Like Transformer From Scratch in PyTorch
  • Word Embeddings: Encoding Lexical Semantics
  • The Magic Behind Embedding Models
  • What are the query, key, and value vectors?
  • Unpacking the Query, Key, and Value of Transformers: An Analogy to Database Operations