Theoretical Machine Learning Course @ ETH Zurich
Lectures MW: High Dimensional Statistics
SS: Understanding Machine Learning
SC: Steinwart and Christmann: Support Vector Machines
| Lecture title | Link to the file | Date | Reference |
|---|---|---|---|
| Introduction and concentration bounds | 26/9/2023 | MW Chapter 2 | |
| Uniform tail bound and McDiarmid | 29/9/2023 | MW Chapter 2,3,4 | |
| Azuma-Hoeffding and the uniform law | 3/10/2023 | MW Chapter 2,4 | |
| Uniform law and Rademacher complexity | 6/10/2023 | SS Chapter 7, 26 | |
| VC bound and margin bounds | 10/10/2023 | SS Chapter 7, 26 | |
| Covering and metric entropy | 17/10/2023 | MW Chapter 5 | |
| Dudley’s integral and chaining | 20/10/2023 | MW Chapter 5, 13 | |
| Non-parametric regression and kernels | 24/10/2023 | SC Chapter4, MW Chapter 12, MW Chapter 13 | |
| Kernel ridge regression | 27/10/2023 | / | |
| Random design | 31/10/2023 | / | |
| Minimax lower bounds | 10/11/2023 | / | |
| Interactive session: Lower bounds for semi-supervised learning | Exercise | 14/11/2023 | / |
| Implicit bias of first-order optimization | pdf | 14/11/2023 | / |
Assignments
| Assignment Link | Answer Link | Offical Answer Link |
|---|---|---|
| Assignment 1 | My Answer 1 | Official Answer 1 |
| Assignment 2 | My Answer 2 | Official Answer 2 |
Project: An Equivalence Between Private Classification and Online Prediction
| Proposal | Final Paper | Presentation |
| Paper Title | Paper Link | Conf / Journal |
|---|---|---|
| Treatment Effect Risk: Bounds and Inference | Management Science | |
| Minimax-Optimal Policy Learning Under Unobserved Confounding | Management Science | |
| When is the estimated propensity score better? High-dimensional analysis and bias correction | / | |
| Counterfactual inference for sequential experiments | / | |
| Effect-Invariant Mechanisms for Policy Generalization | / | |
| Minimax Regret Optimization for Robust Machine Learning under Distribution Shift | COLT 22 | |
| An Algorithmic Framework for Bias Bounties | FACCT 22 | |
| Trained Transformers Learn Linear Models In-Context | / | |
| On Provable Copyright Protection for Generative Models | ICML 23 | |
| Stochastic Bias-Reduced Gradient Methods | NeurIPS 21 | |
| A Universal Law of Robustness via Isoperimetry | NeurIPS 21 | |
| Local Risk Bounds for Statistical Aggregation | COLT 23 | |
| Smoothed Online Learning is as Easy as Statistical Learning | COLT 22 | |
| List-Decodable Robust Mean Estimation and Learning Mixtures of Spherical Gaussians | STOC 18 | |
| Adversarial Resilience in Sequential Prediction via Abstention | / | |
| The One-Inclusion Graph Algorithm is not Always Optimal | / | |
| An Equivalence Between Private Classification and Online Prediction (We selected for project) | FOCS 20 | |
| Understanding the Risks and Rewards of Combining Unbiased and Possibly Biased Estimators, with Applications to Causal Inference | / | |
| Minimax Rates and Adaptivity in Combining Experimental and | ||
| Observational Data | / | |
| Which Invariance Should We Transfer? A Causal Minimax Learning Approach | ICML 23 | |
| On the Value of Target Data in Transfer Learning | NeurIPS 19 | |
| Self-training Converts Weak Learners to Strong Learners in Mixture Models | AISTATS 22 | |
| A Reductions Approach to Fair Classification | ICML 18 | |
| Beyond Adult and COMPAS: Fair Multi-Class Prediction via Information Projection | NeurIPS 22 | |
| Monge, Bregman and Occam: Interpretable Optimal Transport in High-Dimensions with Feature-Sparse Maps | ICML 23 | |
| Online multiple hypothesis testing | / | |
| The difference between structural counterfactuals and potential outcomes | / |
9/10/2023: Add assignment 1 and personal answers for assignment 1; Add Lecture 1 - 4 PDF; Add reference book " High-Dimensional Statistics" by Martion J. Wainwright
11/10/2023: Fix typos and mistakes in assignment 1. Add lecture 5 and interactive session materials.
11/10/2023: Add suggested paper list this semester
17/10/2023: Add Lecture 6 Slides. Reseach proposal of the project: to be added
21/10/2023: Add lecture 7 slide. Research proposal of the project: to be added. Due on 24/10. Especially update the lecture 5 slides since the proof is added.
23/10/2023: Add Project Proposal: An Equivalence Between Private Classification and Online Prediction
21/12/2023: Add Lecture notes, Exercise sheets (finish). Add presentation slides. Final Paper: to be released. Add cheatsheets for oral (referred by Tao Sun's notes)