This document is a prelimnary definition of a curriculum to support development of a applied machine learning program with a focus on engineering for state-of-the-art methods, deep neural networks.
This document serves as an outline for a two-year curriculum with an example schedule, noting prerequesities and summarizing educational content.
Note: Students who already have taken Calculus / Linear Algebra / Probability & Statistics are in a much better position to handle the schedule.
Provides an introudction to machine learning, with a focus on neural networks. Students will cover the foundational mathematics needed to work with and understand model development, and conclude the year focusing on end to end training.
This semester introduces machine learning, learning tasks, models, model selection, in addition to the foundational mathematics of machine learning.
- Topics: High-level coverage of types of learning tasks (supervised, unsupervised, etc.), types of models (neural networks, decision trees, regression, etc.), and introduction to model selection.
- Prerequisites: None
- Topics: Limits and Continuity, Derivatives, Chain Rule, Integrals, Partial Derivatives, Vector Calculus
- Prerequisites: Math placement
- Topics: Systems of linear equations, Matrix operations, Inverses and Determinants, Eigenvalues / Eigenvectors, Matrix Decomposition, SVD, PCA, LDA
- Prerequisites: Math placement
- Topics: Programming languages commonly used, current state-of-the-art frameworks for machine learning.
- Prerequisites: None
This semester continues the mathematical foundations for ML, and introduces students to all apsects of training and building models end to end, in addition to an introduction to a focus on neural networks.
- Topics: Perceptron, Activations, Layers, Cost Function, Back-propegation, Gradient-Descent, Traning: Learning rates / batching, Regularization, Architectures and hyper-parameters
- Prerequisites: Calc, Linear Algebra, Intro to Machine Learning
- Topics: Framing, Problem Structuring, Data Handling / Preprocessing, Model Selection, Training, and Performance Metrics.
- Prerequisites: Intro to Machine Learning
- Topics: Random Variables, Probability Distributions, Marginal / Conditional Probability, Independence, Expectation, Variance and Covariance
- Prerequisites: Calc
This year utilizes the foundations acquired in the first year to focus on deep learning covering specific models and applications, challenges in systems engineering, and completes with a cap-stone project in which the student delivers an ened-to-end state-of-the-art system, preferably in the context of an internship.
This semester focuses on a survey of deep learning and its applications, advanced techniques and focused topics in training deep networks, and systems engineering focused on the computational challenges in the domain of deep learning.
- Topics: CNNs, RNNs, Deep RL/HRL, Transformers & LLMs, AutoEncoders, GANs
- Prerequisites: Intro to Neural Nets, Training End-to-End, Calc, Linear Algebra, Probability & Stats
- Topics: Transfer Learning, Fine-Tuning, Hyper Parameter tuning, Optimizers
- Prerequisites: Calculus, Linear Algebra, Probability & Stats, Intro to Neural Nets, Training End-to-End
- Topics: GPU/TPU acceleration, distributed training, cloud platforms, large datasets, data / model parallelism, MLOps, Versioning, Inference on Edge / constrained compute.
- Prerequisites: Introduction to Neural Networks, Training End-to-End
This semester is focused on applying the foundational knowledge to design and train a state-of-the-art system end to end. Students will also take a survey covering the interdisciplinary applications of machine learning systems in order to gain awareness of industrial demand for machine learning systems. In addition, students will be introduced to the ethical concerns and societal impacts of machine learning and AI.
- Topic: Design and deliver state of the art system.
- Prerequisites: Deep Learning, Systems Engineering
- Topics: Healthcare, Finance, Smart Cities, IoT, Robotics & Autnomous Platforms
- Prerequisites: Intro to Machine Learning
- Topics: Bias and Fairness, Transparency and Accountability, Considerations in data collection
- Prerequisites: Intro to Machine Learning