6.882 Bayesian Modeling and Inference
Spring 2017
Room 56-154
Tuesday, Thursday 2:30–4:00 PM
First class: Tuesday, February 7
Instructor:
Professor Tamara Broderick
Office Hours: Thursday, 4–5pm, 32-G498
Email: 
TA:
Dr. Trevor Campbell
Office Hours: Tuesday, 4–5pm, Location 32-G531
Email: 
Introduction
As both the number of data sets and data set sizes grow, practitioners are interested in learning increasingly complex information and interactions from data. Probabilistic modeling in general, and Bayesian approaches in particular, provide a unifying framework for flexible modeling that includes prediction, estimation, and coherent uncertainty quantification. In this course, we will cover the modern challenges of Bayesian inference, including (but not limited to) speed of approximate inference, making use of distributed architectures, streaming data, and complex data interactions. We will study Bayesian nonparametric models, wherein model complexity grows with the size of the data; this allows us to learn, e.g., a greater diversity of topics as we read more documents from Wikipedia, identify more friend groups as we process more of Facebook's network structure, etc.
Piazza Site
Our course Piazza page is here and will be updated shortly: https://piazza.com/mit/spring2017/6882
Description
This course will cover Bayesian modeling and inference at an advanced graduate level. A tentative list of topics (which may change depending on our interests) is as follows:
- Introduction to Bayesian inference; motivations from de Finetti, decision theory, etc.
- Hierarchical modeling, including popular models such as latent Dirichlet allocation
- Approximate posterior inference
- Variational inference, mean-field, stochastic variational inference, challenges/limitations of VI, etc
- Monte Carlo, avoiding random-walk behavior, Hamiltonian Monte Carlo/NUTS/Stan, biasing, etc
- Evaluation, sensitivity, robustness
- Bayesian nonparametrics: why and how
- Mixture models, admixtures, Dirichlet process, Chinese restaurant process
- Feature allocations, beta process, Indian buffet process
- Combinatorial stochastic processes
- Learning functions, Gaussian processes
- Probabilistic numerics
- Bayesian optimization
Prerequisites
Requirements: 6.867 or a more advanced graduate-level machine learning course (6.437, 6.438 are great), 6.041B or more advanced probability background, 18.06 or more advanced linear algebra background. Or see instructor for permission.
A graduate-level familiarity with statistics, machine learning, and probability is required. We will assume familiarity with graphical models, exponential families, finite-dimensional Gaussian mixture models, expectation maximization, linear & logistic regression, hidden Markov models.
Assessment
- Project
- A project proposal and proposal interview will be due in the first half of the semester.
- A project final report and presentation will be due at the end of the semester.
- Potential project ideas and more details on the project coming soon.
- Presentation and Participation
- There will be assigned reading (typically research papers) each week.
- Students will take turns presenting and leading the discussion in class.
- Students will submit a weekly reflection on the reading (<= 1 page) before class. There will be guiding questions for the reflection and discussion.
- Scribe
- Students will take turns scribing notes from lectures. A template will be provided.