/bml-course

Public repo for course material on Bayesian machine learning at ENS Paris-Saclay and Univ Lille

Primary LanguageJupyter Notebook

Bayesian machine learning

Lecturers

Material

  • Annotated slides on Bayesics are here and here.
  • Annotated slides on MCMC are here and here.
  • Incomplete and drafty lecture notes are available in the notes folder. Any comment welcome, live or as a raised issue.
  • Practicals and exercises are available in the corresponding folders. They are to be done on a voluntary basis. Solutions will be provided on demand.

Objective of the course

By the end of the course, the students should

  • have a high-level view of the main approaches to making decisions under uncertainty.
  • be able to detect when being Bayesian helps and why.
  • be able to design and run a Bayesian ML pipeline for standard supervised or unsupervised learning.
  • have a global view of the current limitations of Bayesian approaches and the research landscape.
  • be able to understand the abstract of most Bayesian ML papers.

Topics

  • Decision theory
  • 50 shades of Bayes: Subjective and objective interpretations
  • Bayesian supervised and unsupervised learning
  • Bayesian computation for ML: Advanced Monte Carlo and variational methods
  • Bayesian nonparametrics
  • Bayesian methods for deep learning

Prerequisites

  • An undergraduate course in probability.
  • It is recommended to have followed either "Probabilistic graphical models" or "Computational statistics" during the first semester.

Organization of courses

  • 8x3 hours of lectures, the last session being a student seminar.
  • All classes and material will be in English. Students may write their final report either in French or English.

Validation

  • Students form groups. Each group reads and reports on a research paper from a list. We strongly encourage a dash of creativity: students should identify a weak point, shortcoming or limitation of the paper, and try to push in that direction. This can mean extending a proof, implementing another feature, investigating different experiments, etc.
  • Deliverables are a small report and a short oral presentation in front of the class, in the form of a student seminar, which will take place during the last lecture.
  • "Auditeurs libres" who need a grade will be given a different assignment, depending on their situation.

References

  • Parmigiani, G. and Inoue, L. 2009. Decision theory: principles and approaches. Wiley.
  • Robert, C. 2007. The Bayesian choice. Springer.
  • Murphy, K. 2023. Probabilistic Machine Learning: Advanced Topics. MIT Press. pdf available at this link.
  • Ghosal, S., & Van der Vaart, A. W. 2017. Fundamentals of nonparametric Bayesian inference. Cambridge University Press.