David Bau


MIT CSAIL
32 Vassar Street, 32-383
Cambridge, MA 02139
people.csail.mit.edu/davidbau
davidbau@csail.mit.edu
+1-781-296-9825

Research Interests

Computer vision, Explanation of deep networks, HCI.

Research Projects

GAN Dissection, gandissect.csail.mit.edu. An analysis method that revals emergent object concepts represented in the middle layers of a GAN (trained without supervision of labels). The encoding of objects is simple enough that objects can be added or removed from a scene by activating or silencing units in the GAN directly. I apply this technique to semantic photo manipulation in GAN Paint, ganpaint.io.
Seeing What a GAN Cannot Generate, ganseeing.csail.mit.edu. A method for measuring and visualizing mode dropping in state-of-the-art GANs. This work reveals that GANs omissions can drop object classes when synthesizing a scene.
Network Dissection, netdissect.csail.mit.edu. A system that quantifies human-interpretable concept detectors within representations of deep networks for vision. This work was used to identify emergent semantics in a range of settings, and to quantify the disentanglement of meaningful individual units in vision networks.

Education

2015-present
Massachusetts Institute of Technology, Cambridge, MA
Ph.D. Candidate in Electrical Engineering and Computer Science
Thesis topic: The Representation of Visual Concepts in Deep Networks for Vision
Advisor: Antonio Torralba
Anticipated graduation: June 2020
1992-1994
Cornell University, Ithaca, NY
M.S. in Computer Science
Book coauthored: Numerical Linear Algebra
Advisor: Lloyd N. Trefethen
1988-1992
Harvard College, Cambridge, MA
A.B. in Mathematics

Awards

MIT EECS Great Educators Fellowship, 2015
NSF Graduate Research Fellowship, 1992

Employment

2013-2015
Pencil Code. pencilcode.net. With Google and open-source contributors.
2009-2014
Google Image Search. images.google.com. Staff software engineer.
2007-2008
Google Search. www.google.com. Staff software engineer.
2004-2007
Google Talk. talk.google.com (now known as Hangouts). Staff software engineer.
2003
XML Beans. Contributor to the Apache Foundation.
2000-2003
Weblogic Workshop. Crossgain and BEA Systems.
1993-2000
Microsoft. Several projects:

Peer-Reviewed Publications

Conference papers

David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, and Antonio Torralba. Seeing What a GAN Cannot Generate. Proceedings of the IEEE International Conference on Computer Vision, pp. 4502-4511. (ICCV 2019 oral presentation)
David Bau, Hendrik Strobelt, William Peebles, Jonas Wulff, Bolei Zhou, Jun-Yan Zhu, and Antonio Torralba. Semantic Photo Manipulation with a Generative Image Prior. ACM Transactions on Graphics (TOG) 38, no. 4. (SIGGRAPH 2019)
Didac Suris, Adria Recasens, David Bau, David Harwath, James Glass, and Antonio Torralba. Learning words by drawing images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. (CVPR 2019)
David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B. Tenenbaum, William T. Freeman, and Antonio Torralba. GAN Dissection: Visualizing and Understanding Generative Adversarial Networks. Proceedings of the Seventh International Conference on Learning Representations. (ICLR 2019)
Leilani H. Gilpin, David Bau, Ben Z. Yuan, Ayesha Bajwa, Michael Specter, Lalana Kagal. Explaining Explanations: An Overview of Interpretability of Machine Learning. Proceedings of the IEEE 5th International Conference on Data Science and Advanced Analytics. (DSAA 2018)
Bolei Zhou, Yiyou Sun, David Bau, and Antonio Torralba. Interpretable Basis Decomposition for Visual Explanation. Proceedings of the European Conference on Computer Vision. (ECCV 2018)
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, Antonio Torralba. Network Dissection: Quantifying Interpretability of Deep Visual Representations. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017 oral presentation)
David Bau, Matt Dawson M, Anthony Bau, C.S. Pickens Pencil Code: Block Code for a Text World. Proceedings of the 14th International Conference on Interaction Design and Children. pp 445-448. (IDC 2015)
Ming Zhao, Jay Yagnik, Hartwig Adam, David Bau. Large Scale Learning and Recognition of Faces in Web Videos. 8th IEEE International Conference on Automatic Face and Gesture Recognition. (FG 2008)
David Bau, Induprakas Kodukula, Vladimir Kotlyar, Keshav Pingali, Paul Stodghill. Solving Alignment Using Elementary Linear Algebra. Languages and Compilers for Parallel Computing, Lecture Notes in Computer Science Volume 892, pp 46-60. (LCPC 1994)

Workshop papers

David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, and Antonio Torralba. Inverting Layers of a Large Generator. ICLR Debugging Machine Learning Models Workshop. (ICLR 2019 workshop)
Jonathan Frankle, David Bau. Dissecting Pruned Neural Networks. ICLR Debugging Machine Learning Models Workshop. (ICLR 2019 workshop)
Saksham Aggarwal, David Anthony Bau, David Bau. A blocks-based editor for HTML code. IEEE Blocks and Beyond Workshop, pp. 83-85. (VL/HCC 2015 workshop)
David Bau, Anthony Bau. A Preview of Pencil Code: A Tool for Developing Mastery of Programming. Proceedings of the 2nd Workshop on Programming for Mobile & Touch. (PROMOTO 2014)

Journals and Magazines

David Bau, Bolei Zhou, Aude Oliva, Antonio Torralba: Interpreting Deep Visual Representations via Network Dissection. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) Volume 41 Issue 9, September 2019, pp. 2131-2145.
David Bau, Jeff Gray, Caitlin Kelleher, Josh Sheldon, Franklyn Turbak. Learnable Programming: Blocks and Beyond. Communications of the ACM (CACM) Volume 60 Issue 6, June 2017. pp. 72-80.

Book

Lloyd N. Trefethen, David Bau. Numerical Linear Algebra. (373pp.) Society for Industrial and Applied Mathematics. (1997)

Selected Patents

David Bau, Google. Predictive hover triggering. US Patent 8621395. (2011)
David Bau, Gunes Erkan, O.A. Osman, Scott Safier, Conrad Lo, Google. Providing Images of Named Resources in Response to a Search Query. US Patent 8538943. (2008)
David Bau, Google. Determining Advertisements Using User Behavior Information Such as Past Navigation Information. WO Patent 2006039393. (2005)
David Bau. Method and System for Anonymous Login for Real Time Communications. US Patent 8725810. (2005)
David Bau, John Perlow, Google. Presenting Quick List of Contacts to Communication Application User US Patent 8392836. (2005)
Rod Chavez, David Bau, Gary Burd, Google. Method and System for Managing Real-time Communications in an Email Inbox. US Patent 8577967. (2005)
Reza Behforooz, Gary Burd, David Bau, John Perlow, Google. Managing Presence Subscriptions for Messaging Services. US Patent 8751582. (2005)
David Bau, Google. User-Friendly Features for Real-Time Communications. US Patent 8095665. (2005)
Kyle Marvin, David Remy, David Bau, Rod Chavez, David Read, BEA Systems. Systems and Methods for Creating Network-Based Software Services Using Source Code Annotations. US Patent 7707564. (2004)
David Bau, BEA Systems. XML Types in Java. US Patent 7650591. (2004)
David Bau, Adam Bosworth, Gary Burd, Rod Chavez, Kyle Marvin, BEA Systems. Annotation Based Development Platform for Asynchronous Web Services. US Patent 7356803. (2002)
Andrei C, Adam Bosworth, David Bau, BEA Systems. Declarative Specification and Engine for Non-Isomorphic Data Mapping. US Patent 6859810. (2001)
Adam Bosworth, David Bau, K. Eric Vasilik, Oracle. Multi-Language Execution Method. US Patent 7266814. (2001)
Adam Bosworth, David Bau, K. Eric Vasilik, Oracle. Cell Based Data Processing. US Patent 8312429. (2000)

Invited Talks

Dissecting the Semantic Structure of Deep Networks for Vision. Explainable AI for Vision Workshop. Seoul, Korea. November 2019.
Dissecting and Manipulating Generative Adversarial Networks. Image Synthesis Workshop. Seoul, Korea. October 2019.
Understanding the Internal Structure of a GAN. Re-Work Deep Learing Summit. Boston, MA. May 2019.
Dissecting Artificial Neural Networks for Vision. Martinos Center for Biomedical Imaging. Boston, MA. April 2019.
Semantic Paint using a Generative Adversarial Network. Samsung/MIT Design Workshop. Cambridge, MA. April 2019.
Dissecting What a Generative Network Can Learn Unsupervised. DARPA XAI PI Meeting. Berkeley, CA. February 2019.
Interpretation of Deep Networks for Vision. Trustworthy and Robust AI Initiative. Cambridge, MA. February 2019.
Visualizing and Understanding Generative Adversarial Networks. AAAI Workshop on Network Interpretability. Honolulu, HI. January 2019.
Explaining Explanations: Interpretation of Deep Neural Networks. Trust.ML Workshop on Public Policy Aspects of ML. Cambridge, MA. June 2018.

Organized Workshops

Structure and Intpretation of Deep Networks, Workshop organizer.
Cambridge, MA. January 2020.
Explainable AI for Vision Workshop, Workshop organizer.
Seoul Korea, November 2019.
Robust and Interpretable Deep Learning Symposium, Workshop organizer.
Cambridge, MA. November 2018.
Blocks and Beyond, Workshop organizer.
Memphis, TN. July 2017.
Coding Projects for Humanities Classes, Workshop organizer.
Cambridge, MA. May 2014.
Pencil Code CS Teaching Hackathon, Workshop organizer.
Cambridge, MA. March 2014.
Teaching with Pencil Code, Workshop organizer.
Cambridge, MA. February 2014.

Students Supervised

Masters Theses

Brian Shimanuki. Joint GAN generation of text and images.
Richard Yip. Understanding What a Captioning Network Doesn't Know.

Undergraduate Research

James Gilles. Analysis of representation similarity across vision networks.
Wendy Wei. Visualization of semantic clusters in a population of networks.
Kaveri Nadhamuni. A search for bug-causing neurons in classifiers.
William Peebles. Semantic manipulation of a user-provided photo.
Steven Liu. Self-conditioned Generative Adversarial Networks.
Tony Peng. Using GANs to alter the lighting in a scene.

Other Activities

Lincoln Middle School Math Team Coach. 2009-2015.
Lincoln Gear Ticks FLL Robotics Coach. 2012-2013.

References

Prof. Antonio Torralba
MIT CSAIL
32 Vassar Street
Cambridge, MA 02139
torralba@mit.edu
Prof. William T. Freeman
MIT CSAIL
32 Vassar Street
Cambridge, MA 02139
billf@mit.edu
Prof. Joshua B. Tenenbaum
Brain and Cognitive Sciences
MIT
77 Massachusetts Ave, 46-4015
Cambridge, MA 02139
jbt@mit.edu
Prof. Alexei Efros
EECS Department
UC Berkeley
395 Soda Hall #1776
Berkeley, CA 94720
efros@eecs.berkeley.edu
Prof. Phillp Isola
MIT CSAIL
32 Vassar Street
Cambridge, MA 02139
phillipi@mit.edu
Aude Oliva
Principal Research Scientist
MIT CSAIL
32 Vassar Street
Cambridge, MA 02139
oliva@mit.edu