florent-leclercq/Bayes_InfoTheory

Lectures on Bayesian statistics and information theory

Bayes_InfoTheory

This is a repository of Jupyter notebooks used by Florent Leclercq during lectures on Bayesian statistics and Information Theory. The homepage of the lectures is accessible here.

Contents

• Ignorance priors (exemplified with the lighthouse problem) and the maximum entropy principle
• Gaussian random fields:
  • Examples and a digression on non-Gaussianity
  • Bayesian signal processing and reconstruction: de-noising
  • Bayesian signal processing and reconstruction: de-blending
• Bayesian decision theory
• Markov Chain Monte Carlo:
  • Monte-Carlo integration, importance sampling, rejection sampling
  • Metropolis-Hastings algorithm & Gelman-Rubin test (MCMC)
  • Slice sampling
  • Gibbs sampling
  • Hamiltonian sampling (HMC), and comparison of 2nd and 4th order integrators for HMC
• Approximate Bayesian Computation:
  • Likelihood-free rejection sampling
  • Synthetic likelihood (parametric approximation)
  • Discrepancy and effective likelihood (non-parametric approximation)
• Information theory:
  • The noisy binary symmetric channel
  • Supervised Machine Learning basics: Titanic example
• Cosmological and physical examples:
  • Wiener filtering for the Cosmic Microwave Background
  • Bayesian decision theory for Cosmic Web classification
  • Supernova cosmology: data and simulations (preliminary exercise) and inference with MCMC and HMC
  • The 1919 Eclipse: parameter inference and model comparison

Acknowledgments

I thank Benjamin Wandelt for his own lectures, which have inspired a fraction of this material, and the SOC/LOC of the ICIC Data Analysis workshop 2021 and STFC Summer School on Data Intensive Science 2021.

Warranty

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.