/bayes-nn

Lecture notes on Bayesian deep learning

Understanding Bayesian Deep Learning

1. Elementary mathematics

  • Set theory
  • Measure theory
  • Probability
  • Random variable
  • Random process
  • Functional analysis (harmonic analysis)

2. Gaussian process

  • Gaussian process
  • Weight-space view
  • Function-space view
  • Gaussian process latent variable model

3. Bayesian neural netwrok

  • Minimum description length
  • Ensemble learning in Bayesian neural network
  • Practical variational inference
  • Bayes by backprop
  • Summary of variational inference
  • Dropout as a Bayesian approximation
  • Stein variational gradient descent

4. Summary

  • Measure thoery
  • Probability
  • Random variable
  • Random process
  • Gaussian process
  • Functional Analysis
  • Summary of variational inference
  • Stein variational gradient descent

5. Uncertainty in Deep Learning

  • Yarin Gal, Uncertainty in Deep Learning
  • Anonymous, Bayesian Uncertainty Estimation for Batch Normalized Deep Networks
  • Patrick McClure, Representing Inferential Uncertainty in Deep Neural Networks through Sampling
  • Balaji Lakshminarayanan, Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
  • Alex Kendal, What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
  • Gregory Kahn, Uncertainty-Aware Reinforcement Learning for Collision Avoidance
  • Charles Richter, Safe Visual Navigation via Deep Learning and Novelty Detection
  • Sungjoon Choi, Uncertainty-Aware Learning from Demonstration Using Mixture Density Networks with Sampling-Free Variance Modeling