6.883 Science of Deep Learning: Bridging Theory and Practice -- Spring 2018

Konstantinos Daskalakis (costis@csail.mit.edu), 32-G694
Aleksander Mądry (madry@mit.edu), 32-G666
Teaching Assistants:
Andrew Ilyas (ailyas@mit.edu)
Dimitris Tsipras (tsipras@mit.edu)
Emmanouil Zampetakis (mzampet@mit.edu)
Time and place: Mondays and Wednesdays 2:30-4 pm in 54-100
Units: 3-0-9 (H level)
Algorithms (6.046 or similar); probability (6.042/6.041/6.008 or similar); ML (6.867 or similar); or permission of instructor

Course description

Recent advances in deep learning have enabled us to make tremendous progress on a number of tasks in machine learning, computer vision, and robotics. However, a principled understanding of the roots of this success – as well as why and to what extent deep learning works – still eludes us.

This course will aim to cover fundamental ideas and phenomena that underlie recent developments in deep learning. We will explore topics revolving around optimization landscape of neural network training; generalization of deep learning models; generative models such as generative adversarial networks (GANs) and variational autoencoders (VAEs); adversarial aspects of machine learning; interpretability, robustness and privacy of deep learning models; and (deep) reinforcement learning.

The lectures will be a mix of surveying some of the recent advances in the field of deep learning and of presenting topics in optimization, learning theory, statistics and game theory that are relevant in this context. The presented material will provide a starting point for a subsequent class discussion on the merits and shortcomings of the presented state-of-the-art.

Class projects will aim to address some of these identified shortcomings. The focus will be on building a principled understanding of deep learning via a mixture of empirical evaluations and theoretical modeling. The projects are intended to serve as a starting point for a subsequent publication in an ML conference.

Lecture notes:
[Remaining notes coming soon.]       (Deep) ML Fundamentals:       (Deep) Generative Models:       Robust ML:       (Deep) Reinforcement Learning:       Societal Impact of ML: