6.883 Science of Deep Learning: Bridging Theory and Practice -- Spring 2018
Instructors: Konstantinos Daskalakis (costis@csail.mit.edu), 32-G694Aleksander Mądry (madry@mit.edu), 32-G666
Teaching Assistants: Andrew Ilyas (ailyas@mit.edu)
Dimitris Tsipras (tsipras@mit.edu)
Emmanouil Zampetakis (mzampet@mit.edu)
Time and place: Mondays and Wednesdays 2:30-4 pm in 54-100
Units: 3-0-9 (H level)
Prerequisites
Course description
Recent advances in deep learning have enabled us to make tremendous progress on a number of tasks in machine learning, computer vision, and robotics. However, a principled understanding of the roots of this success – as well as why and to what extent deep learning works – still eludes us.
This course will aim to cover fundamental ideas and phenomena that underlie recent developments in deep learning. We will explore topics revolving around optimization landscape of neural network training; generalization of deep learning models; generative models such as generative adversarial networks (GANs) and variational autoencoders (VAEs); adversarial aspects of machine learning; interpretability, robustness and privacy of deep learning models; and (deep) reinforcement learning.
The lectures will be a mix of surveying some of the recent advances in the field of deep learning and of presenting topics in optimization, learning theory, statistics and game theory that are relevant in this context. The presented material will provide a starting point for a subsequent class discussion on the merits and shortcomings of the presented state-of-the-art.
Class projects will aim to address some of these identified shortcomings. The focus will be on building a principled understanding of deep learning via a mixture of empirical evaluations and theoretical modeling. The projects are intended to serve as a starting point for a subsequent publication in an ML conference.
[Remaining notes coming soon.] (Deep) ML Fundamentals:
- (2/12) Overview of Continuous Optimization Methods
- (2/14) Overview of the Theory of Generalization
- (2/20) Optimization Landscape of Deep Learning
- (2/21) Generalization in Deep Learning
- (2/26) Towards Understanding SGD
- (2/28) Overview of (Deep) Generative Models
- (3/5) Game-theoretic View on GAN Dynamics
- (3/7) Are GANs Truly Distribution Learners?
- (3/12) Distribution Testing
- (3/14) Adversarial Examples and Misclassification Attacks I
- (3/19) Adversarial Examples and Misclassification Attacks II
- (3/21) Adversarially Robust Generalization
- (4/2) Data Poisoning and Differential Privacy
- (4/9) Introduction to Reinforcement Learning
- (4/11) Monte Carlo Tree Search and AlphaGo
- (4/18) Policy Gradient and Actor-Critic Methods
- (4/23) Exploration in Reinforced Learning
- (4/25) Challenges of Deep Reinforcement Learning Research
- (4/30) Fairness in ML
- (5/2) Interpretability in ML
- (5/7) Causality in ML
- (5/9) No lecture
- (5/14) No lecture
- (5/16) AI Safety and AI Alignment