layout | title | nav_exclude | permalink | seo | ||||
---|---|---|---|---|---|---|---|---|
home |
Probabilistic ML course |
true |
index.html |
|
{: .no_toc .text-delta }
- TOC {:toc}
u7jjtu2
- Mathematics for ML: Recommended reading is MML book
- A prior ML course: ES654 course at IIT Gandhinagar or equivalent
- Good programming skills in Python. Familiarity with: numpy, Pandas, matplotlib, sklearn. Recommended reading is Python Data Science Handbook
- Probability refresher: Probability theory, discrete distributions, continuous distributions, joint probability distributions, sampling from different distributions (e.g. using Box-Muller transform), uncertainty modelling, information theoretic concepts: (KL-divergence, entropy)
- Bayesian concept learning: Likelihood, Prior, Posterior, Maximum Likelihood Estimate (MLE), Maximum A-Posteriori estimation (MAP), Full Bayesian Estimation with Conjugate Priors (Beta-Bernoulli model for the coin toss example, Normal-Normal model for estimating parameters of Normal distribution)
- Bayesian supervised methods:
- Regression: Bayesian linear regression, Robust linear regression via alternative likelihood (e.g. Laplace, Student-T)
- Classification: Bayesian Logistic regression, Bayesian Naive Bayes
- Latent Variable modelling: Gaussian Mixture Model, Probabilistic principal component analysis (PPCA)
- Approximate Inference:
- Sampling based strategies: Rejection sampling, Importance sampling, Markov Chain Monte Carlo (MCMC), Metropolis Hastings, Gibbs sampling, No U-Turn sampler (NUTS)
- Variational inference: Mean field approach, Evidence Lower Bound (ELBO), Reparameterization trick, Stochastic Variational Inference, Automatic Differentiation Variational Inference (ADVI)
- Gaussian Processes (GP): Multivariate Normal distribution and its properties, Kernels, GP regression, GP classification, Approximate Inference and Sparse GPs
- Bayesian Optimization (BO) and Active Learning (AL): AL - Query by committee, Uncertainty sampling, Expected model change, BO - Acquisition functions, GP based BO, Random Forests based BO
- Probabilistic Deep Learning: MC Dropout, Deep Ensembles, Bayesian neural networks (BNNs), Deep GPs
- Kevin Murphy. Machine Learning, A Probabilistic Perspective. The MIT Press, 2012.
- Kevin Murphy. Probabilistic Machine Learning: An Introduction. The MIT Press, 2022
- Kevin Murphy. Probabilistic Machine Learning: Advanced Topics. The MIT Press, 2023.
- Chris Bishop. Pattern Recognition and Machine Learning.
- Allen Downey. Think Bayes: Bayesian Statistics in Python. Green Tea Apress, 2012
- David Barber. Bayesian Reasoning and Machine Learning. Cambridge University Press, 2012
- Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2006
- Richard McLearth. Statistical Rethinking. CRC Press, 2020.
- PML course from Piyush Rai (IITK)
- Assignments: 20%
- Quizzes: 20%
- Quiz syllabus will be everything from previous quiz to present day
- Projects: 60%