For those who may be tempted to believe that I had at any point signed off on the outrageous claim that GPT-4 achieves a perfect solve rate on
the entire EECS curriculum, I want to point out that when Iddo posted the paper on arXiv without my permission, he actually left more or less
intact the introduction that I had originally written for the version submitted to the NeurIPS Datasets and Benchmarks track.
It wasn't my best writing (it was a draft and not yet meant for publication), but it reflects my understanding of the paper I
had signed off to submit. You will note that it makes no mention of the claim that GPT-4 can
solve the entire curriculum. It was meant to be a dataset paper about a really compelling dataset and some of the neat things you can do with it,
such as understanding dependencies between courses. Even that paper, I now know, should never have been submitted due to the unethical
sourcing of the data. But I can assure everyone that if I ever want to submit a paper that claims that I can automatically ace the entire MIT
curriculum, that calim will be stated prominently in the introduction of the paper, and the paper will not be submitted to
a Datasets and Benchmarks track. What's the point of a benchmark if the state of the art tool can already solve 100% of the problems?
I lead the
Assisted Programming Group
. The focus of our research is program synthesis, an exciting
research area that lies at the intersection of Programming Systems and Artificial Intelligence.
On the one hand, program synthesis is about the use of automated reasoning and learning to help
bring more automation to the programming process. On the other hand, we believe code provides
a uniquely versatile modeling mechanism, so program synthesis can play a powerful role in helping
to build learning systems that are more predictable and robust.
Over the last few years, I have helped with the development of a few courses:
Program Synthesis: Everything you wanted to know about the state of the
art in program synthesis, especially the techniques developed by the
programming languages and formal methods communities.
Here are some lecture notes I put together while on sabbatical in Mexico City in
6.820--Foundations of Program Analysis: This is an introduction to many of the
core techniques that one can use to reason about programs, including
different forms of semantics, type systems, program logics, abstract interpretation
and model checking. And you also learn Haskell along the way!
The course is offered every 2 years. The class was last offered in
6.s081-- Dynamic Computer Language Engineering:
has been mostly leading this class, but I helped with the initial version
of it. In this class, students get to build a just-in-time compiler for
a non-trivial dynamic language completely from scratch.
See here for
a recent version of this class.
In recent years, I have also been involved in teaching
6.031 (Software Construction),
6.009 (Fundamentals of Programming) and
6.01 (Introduction to EECS via Robotics).
In my free time, I created PlaySkript, an online platform that I use for creating
and supports both direct manipulation and automated refactoring in order to help create complex presentations more easily.
Try it out!