Overview

Overview#

Instructor: Scott Linderman
TA: Xavier Gonzalez
Term: Spring 2023
Stanford University

Course Description#

Probabilistic modeling and inference of multivariate data. Topics may include multivariate Gaussian models, probabilistic graphical models, MCMC and variational Bayesian inference, dimensionality reduction, principal components, factor analysis, matrix completion, topic modeling, and state space models. Extensive work with data involving Python programming using PyTorch.

Prerequisites#

Students should be comfortable with probability and statistics as well as multivariate calculus and linear algebra. This course will emphasize implementing models and algorithms, so coding proficiency is required.

Logistics#

Time: Tuesday and Thursday, 10:30-11:50am
Level: advanced undergrad and up
Grading basis: credit or letter grade
Office hours:
- Weds 4:30-5:30pm in Wu Tsai Neurosciences Instiute Room M252G (Scott)
- Thurs 5-7pm location Wu Tsai Neurosciences Instiute Room S275 (Xavier)
Assignments released Friday, due the following Thursday at 11:59pm

Books#

We will primarily use

Murphy. Probabilistic Machine Learning: Advanced Topics. MIT Press, 2023. link

You may also find these texts helpful

Bishop. Pattern recognition and machine learning. New York: Springer, 2006. link
Gelman et al. Bayesian Data Analysis. Chapman and Hall, 2005. link `

Schedule#

Date	Topic	Reading
Apr 4	Bayesian Analysis of the Normal Distribution `[slides]` [notebook]	Ch 2.3, 3.4.3
Apr 6	Multivariate Normal Distribution `[slides]` [notebook]	Ch 2,3, 3.4.4
Apr 11	Probabilistic Graphical Models `[slides]` [notebook]	Ch 3.6.2, 4.2
Apr 13	Markov Chain Monte Carlo `[slides]` [notebook]	Ch 11.1-11.2, 12.1-12.3
Apr 18	Probabilistic PCA and Factor Analysis `[slides]`	Ch 28.3
Apr 20	Hamiltonian Monte Carlo `[slides]`	Neal, 2012, Ch 12.5
Apr 25	Mixture Models `[slides]`	Ch 28.2
Apr 27	Expectation Maximization `[slides]`	Ch 6.5
May 2	Coordinate Ascent Variational Inference `[slides]` [notebook 1] [notebook 2]	Blei et al, 2017, Ch 10.1-10.3
May 4	Mixed Membership Models `[slides]` [notebook]	Ch 28.5
May 9	Gradient-Based Variational Inference `[slides]` [notebook]	Ch 21
May 11	Variational Autoencoders `[slides]`	Kingma and Welling, 2019, Ch 10.2
May 16	Hidden Markov Models `[slides]`	Ch 29.2-29.5
May 18	Linear Dynamical Systems `[slides]`	Ch 29.6-12.8
May 23	Gaussian Processes `[slides]`	Ch 18
May 25	Poisson Processes `[slides]`
May 30	Gaussian Processes (continued) `[slides]`
June 1	Dirichlet Process Mixture Models `[slides]`
June 6	Model Comparison and Criticism `[slides]`
June 13	Final Exam `[practice exam]` `[solutions]` `[reference]`