STATS305C: Applied Statistics III

Instructor: Scott Linderman
TA: Matt MacKay, James Yang
Term: Spring 2022
Stanford University

Course Description:

Probabilistic modeling and inference of multivariate data. Topics may include multivariate Gaussian models, probabilistic graphical models, MCMC and variational Bayesian inference, dimensionality reduction, principal components, factor analysis, matrix completion, topic modeling, and state space models. Extensive work with data involving programming, ideally in Python.

Prerequisites:

Students should be comfortable with probability and statistics as well as multivariate calculus and linear algebra. This course will emphasize implementing models and algorithms, so coding proficiency is required.

Logistics:

Time: Monday and Wednesday, 11:30am-1pm
Level: advanced undergrad and up
Grading basis: credit or letter grade
Office hours:
- Monday 1-2pm (Scott)
- Tuesday 5:30-7pm in Bowker, Room 207, Sequoia Hall and over Zoom (Matt)
- Friday 1-2:30pm Zoom (James)
Final evaluation: Exam

Books

Bishop. Pattern recognition and machine learning. New York: Springer, 2006. link
Murphy. Probabilistic Machine Learning: Advanced Topics. MIT Press, 2023. link
Gelman et al. Bayesian Data Analysis. Chapman and Hall, 2005. link

Assignments

Assignment 1: Bayesian Linear Regression. Due Weds. Apr 6 at 11:59pm on GradeScope.
Assignment 2: Gibbs Sampling and Metropolis-Hastings. Due Weds. Apr 13 at 11:59pm on GradeScope.
Assignment 3: Continuous Latent Variable Models. Due Weds. Apr 20 at 11:59pm on GradeScope.
Assignment 4: Bayesian Mixture Models. Due Weds. Apr 27 at 11:59pm on GradeScope.
Assignment 5: Poisson Matrix Factorization. Due Weds. May 4 at 11:59pm on GradeScope.
Assignment 6: Neural Networks and VAEs. Due Weds. May 11 at 11:59pm on GradeScope.
Assignment 7: Hidden Markov Models. Due Weds. May 18 at 11:59pm on GradeScope.

Schedule

Week 1 (3/28 & 3/30): Multivariate Normal Models and Conjugate Priors

Required Reading: Bishop, Ch 2.3
Optional Reading: Murphy, Ch 2.3 and 3.2.4

Week 2 (4/4 & 4/6): Hierarchical Models and Gibbs Sampling

Required Reading: Bishop, Ch 8.1-8.2 and 11.2-11.3
Optional Reading: Murphy, Ch 3.5.2, 4.2, and 11.1-11.3
Optional Reading: Gelman, Ch 5

Week 3 (4/11 & 4/13): Continuous Latent Variable Models and HMC

Required Reading: Bishop, Ch 12.1-12.2
Required Reading: MCMC using Hamiltonian dynamics Neal, 2012

Week 4 (4/18 & 4/20): Mixture Models and EM

Required Reading: Bishop, Ch 9
Optional Reading: Murphy, Ch 6.7

Week 5 (4/25 & 4/27): Mixed Membership Models and Mean Field VI

Required Reading: "Probabilistic topic models" Blei, 2012
Required Reading: "Variational Inference: A Review for Statisticians” Blei et al, 2017
Optional Reading: Murphy, Ch 10.2

Week 6 (5/2 & 5/4): Variational Autoencoders and Fixed-Form VI

Required Reading: “An Introduction to Variational Autoencoders” (Ch 1 and 2) Kingma and Welling, 2019
Optional Reading: Murphy, Ch 10.3

XiangP/stats305c