CV for David Bau

Northeastern University
Khoury College of Computer Sciences
440 Huntington Avenue
Boston, MA 02115

Email:	davidbau@northeastern.edu
Phone:	+1-781-296-9825
Website:	https://baulab.info
Orcid ID:	0000-0003-1744-6765

Research Areas

Computer vision, Explainable machine learning, Human-computer iteraction.

Research Projects

Model Rewriting, rewriting.csail.mit.edu. A project to investigate how a user can directly change the parameters of a deep model according to their own intentions, rather than using a data set for retraining. We find that the weights of a model are structured like an optimal linear associative memory, and we use this insight to develop a method and tool for rewriting the rules of start-of-the-art generative networks.

GAN Paint, gandissect.csail.mit.edu. An analysis method that reveals emergent object concepts represented in the middle layers of a GAN (trained without supervision of labels). The encoding of objects is simple enough that objects can be added or removed from a scene by activating or silencing units in the GAN directly. We apply this technique to semantic photo manipulation in GAN Paint, ganpaint.io.

Network Dissection, dissect.csail.mit.edu. A system that quantifies human-interpretable concept detectors within representations of deep networks for vision. This work is used to identify emergent semantics in a range of settings, and to quantify the disentanglement of meaningful individual units in vision networks.

Education

Massachusetts Institute of Technology, Cambridge, MA
Ph.D. in Electrical Engineering and Computer Science
Thesis: Dissection of Deep Neural Networks
Advisor: Antonio Torralba

Cornell University, Ithaca, NY
M.S. in Computer Science
Book coauthored: Numerical Linear Algebra
Advisor: Lloyd N. Trefethen

Harvard College, Cambridge, MA
A.B. in Mathematics

Awards

MIT EECS Great Educators Fellowship, 2015

NSF Graduate Research Fellowship, 1992

Employment

Assistant Professor. Northeastern Khoury College of Computer Science.

Postdoctoral Fellow. Martin Wattenberg lab, Harvard University.

Research Assistant. Antonio Torralba lab, MIT CSAIL.

Pencil Code. pencilcode.net. With Google and open-source contributors.

Created an educational programming system and curriculum for beginners, with 159,000 active accounts and 2,000 users every school day.

Google Image Search. images.google.com. Staff software engineer.

Image Search Freshness. Led a team of engineers to improve freshness of Google Image Search: added the ability to satisfy queries for new events and new concepts with new and newsworthy results within minutes of their publication on the web.
Image Search Visual Redesign. Led a team to prototype and implement a major image search redesign, providing a long scrolling page of results. Solved infrastructure challenges posed by serving results so quickly at Google's enormous scale.
Facets for Images. Led a team of engineers and designers to implement an image clustering feature at the top of Google Image search.

Google Search. www.google.com. Staff software engineer.

Developed algorithms for Google's processing of queries and web pages about people.

Google Talk. talk.google.com (now known as Hangouts). Staff software engineer.

Created Google's realtime communications product, including text chat, streaming voice over ip, and integration into Gmail. Created the team, recruited engineers, and contributed to all aspects of the technology.

XML Beans. xmlbeans.apache.org Contributor to the Apache Foundation.

Created an open-source compiler for the XML Schema standard in Java.

Weblogic Workshop. Crossgain and BEA Systems.

Led the design of a development tool to simplify the creation of cloud services for Java programmers. Acquired by BEA Systems.

Microsoft. Several projects:

.Net Platform. Specified and implemented aspects of the Common Language Runtime.
Trident project. Defined and implemented core aspects of the programmable engine for the first Ajax web browser (Internet Explorer 4, 5, 6).
Blackbird platform. Designed features for a network media platform for MSN.
Intentional Programming. Intern on Charles Simonyi's innovative programming tool project.

Peer-Reviewed Publications

Journals

David Bau, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, and Antonio Torralba. Understanding the role of individual units in a deep neural network. Proceedings of the National Academy of Sciences (PNAS), Volume 117, no. 48, December 1 2020, pp. 30071-30078.

David Bau, Bolei Zhou, Aude Oliva, Antonio Torralba: Interpreting Deep Visual Representations via Network Dissection. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) Volume 41 Issue 9, September 2019, pp. 2131-2145.

David Bau, Jeff Gray, Caitlin Kelleher, Josh Sheldon, Franklyn Turbak. Learnable Programming: Blocks and Beyond. Communications of the ACM (CACM) Volume 60 Issue 6, June 2017. pp. 72-80.

Conference papers

Shibani Santurkar, Dimitris Tsipras, Mahalaxmi Elango, David Bau, Antonio Torralba, and Aleksander Madry. Editing a classifier by rewriting its prediction rules. Advances in Neural Information Processing Systems 34. (NeuIPS 2021)

Emma Andrews, David Bau, and Jeremiah Blanchard. From Droplet to Lilypad: Present and Future of Dual-Modality Environments. 2021 IEEE Symposium on Visual Languages and Human-Centric Computing. (VL/HCC 2021)

Sarah Schwettmann, Evan Hernandez, David Bau, Samuel Klein, Jacob Andreas, Antonio Torralba. Toward a Visual Concept Vocabulary for GAN Latent Space Proceedings of the IEEE International Conference on Compputer Vision. (ICCV 2021)

Sheng-Yu Wang, David Bau, Jun-Yan Zhu. Sketch Your Own GAN. Proceedings of the IEEE International Conference on Compputer Vision. (ICCV 2021)

David Bau, Steven Liu, Tongzhou Wang, Jun-Yan Zhu, and Antonio Torralba. Rewriting a Deep Generative Model. Proceedings of the European Conference on Computer Vision. (ECCV 2020 oral)

Lucy Chai, David Bau, Ser-Nam Lim, and Phillip Isola. What makes fake images detectable? Understanding properties that generalize. Proceedings of the European Conference on Computer Vision. (ECCV 2020)

Steven Liu, Tongzhou Wang, David Bau, Jun-Yan Zhu, and Antonio Torralba. Diverse Image Generation via Self-Conditioned GANs. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. (CVPR 2020)

David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, and Antonio Torralba. Seeing What a GAN Cannot Generate. Proceedings of the IEEE International Conference on Computer Vision, pp. 4502-4511. (ICCV 2019 oral presentation)

David Bau, Hendrik Strobelt, William Peebles, Jonas Wulff, Bolei Zhou, Jun-Yan Zhu, and Antonio Torralba. Semantic Photo Manipulation with a Generative Image Prior. ACM Transactions on Graphics (TOG) 38, no. 4. (SIGGRAPH 2019)

Didac Suris, Adria Recasens, David Bau, David Harwath, James Glass, and Antonio Torralba. Learning words by drawing images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. (CVPR 2019)

David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B. Tenenbaum, William T. Freeman, and Antonio Torralba. GAN Dissection: Visualizing and Understanding Generative Adversarial Networks. Proceedings of the Seventh International Conference on Learning Representations. (ICLR 2019)

David Weintrop, David Bau, and Uri Wilensky. The cloud is the limit: A case study of programming on the web, with the web. International Journal of Child-Computer Interaction 20. (IJCCI 2019)

Leilani H. Gilpin, David Bau, Ben Z. Yuan, Ayesha Bajwa, Michael Specter, Lalana Kagal. Explaining Explanations: An Overview of Interpretability of Machine Learning. Proceedings of the IEEE 5th International Conference on Data Science and Advanced Analytics. (DSAA 2018)

Bolei Zhou, Yiyou Sun, David Bau, and Antonio Torralba. Interpretable Basis Decomposition for Visual Explanation. Proceedings of the European Conference on Computer Vision. (ECCV 2018)

David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, Antonio Torralba. Network Dissection: Quantifying Interpretability of Deep Visual Representations. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017 oral presentation)

David Bau, Matt Dawson M, Anthony Bau, C.S. Pickens Pencil Code: Block Code for a Text World. Proceedings of the 14th International Conference on Interaction Design and Children. pp 445-448. (IDC 2015)

Ming Zhao, Jay Yagnik, Hartwig Adam, David Bau. Large Scale Learning and Recognition of Faces in Web Videos. 8th IEEE International Conference on Automatic Face and Gesture Recognition. (FG 2008)

David Bau, Induprakas Kodukula, Vladimir Kotlyar, Keshav Pingali, Paul Stodghill. Solving Alignment Using Elementary Linear Algebra. Languages and Compilers for Parallel Computing, Lecture Notes in Computer Science Volume 892, pp 46-60. (LCPC 1994)

Workshop papers

David Bau, Steven Liu, Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba Horses With Blue Jeans - Creating New Worlds by Rewriting a GAN. 4th Workshop on Machine Learning for Creativity and Design (NeurIPS 2020 Workshop)

David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, and Antonio Torralba. Inverting Layers of a Large Generator. ICLR Debugging Machine Learning Models Workshop. (ICLR 2019 workshop)

Jonathan Frankle, David Bau. Dissecting Pruned Neural Networks. ICLR Debugging Machine Learning Models Workshop. (ICLR 2019 workshop)

Saksham Aggarwal, David Anthony Bau, David Bau. A blocks-based editor for HTML code. IEEE Blocks and Beyond Workshop, pp. 83-85. (VL/HCC 2015 workshop)

David Bau, Anthony Bau. A Preview of Pencil Code: A Tool for Developing Mastery of Programming. Proceedings of the 2nd Workshop on Programming for Mobile & Touch. (PROMOTO 2014)

Book

Lloyd N. Trefethen, David Bau. Numerical Linear Algebra. (373pp.) Society for Industrial and Applied Mathematics. (1997)

Preprints

David Bau, Alex Andonian, Audrey Cui, Yeon-Hwan Park, Ali Jahanian, Aude Oliva, Antonio Torralba. Paint by Word. arxiv.org/abs/2103.10951 (2021)

Selected Patents

David Bau, Google. Predictive hover triggering. US Patent 8621395. (2011)

David Bau, Gunes Erkan, O.A. Osman, Scott Safier, Conrad Lo, Google. Providing Images of Named Resources in Response to a Search Query. US Patent 8538943. (2008)

David Bau, Google. Determining Advertisements Using User Behavior Information Such as Past Navigation Information. WO Patent 2006039393. (2005)

David Bau. Method and System for Anonymous Login for Real Time Communications. US Patent 8725810. (2005)

David Bau, John Perlow, Google. Presenting Quick List of Contacts to Communication Application User US Patent 8392836. (2005)

Rod Chavez, David Bau, Gary Burd, Google. Method and System for Managing Real-time Communications in an Email Inbox. US Patent 8577967. (2005)

Reza Behforooz, Gary Burd, David Bau, John Perlow, Google. Managing Presence Subscriptions for Messaging Services. US Patent 8751582. (2005)

David Bau, Google. User-Friendly Features for Real-Time Communications. US Patent 8095665. (2005)

Kyle Marvin, David Remy, David Bau, Rod Chavez, David Read, BEA Systems. Systems and Methods for Creating Network-Based Software Services Using Source Code Annotations. US Patent 7707564. (2004)

David Bau, BEA Systems. XML Types in Java. US Patent 7650591. (2004)

David Bau, Adam Bosworth, Gary Burd, Rod Chavez, Kyle Marvin, BEA Systems. Annotation Based Development Platform for Asynchronous Web Services. US Patent 7356803. (2002)

Andrei C, Adam Bosworth, David Bau, BEA Systems. Declarative Specification and Engine for Non-Isomorphic Data Mapping. US Patent 6859810. (2001)

Adam Bosworth, David Bau, K. Eric Vasilik, Oracle. Multi-Language Execution Method. US Patent 7266814. (2001)

Adam Bosworth, David Bau, K. Eric Vasilik, Oracle. Cell Based Data Processing. US Patent 8312429. (2000)

Invited Talks

Interpretable Deep Learning. Invited Lecture, Brown University Department of Computer Science. December 2021.

Mathematical Puzzles in Intepretable Deep Learning. Computational Maths and Applications Seminar, University of Oxford. October 2021.

Opening Up AI For Human Insight and Creativity. Keynote for Workshop on Measurements of Machine Creativity, at CVPR June 2021.

Cracking Open AI for New Insights. Keynote for Workshop on Analysis and Modeling of Faces, at CVPR June 2021.

Analyzing the Role of Neurons in an Artificial Neural Network. Kanwisher Lab Meeting, MIT Dept of Brain and Cognitive Sciences. Cambridge, MA. September 2020.

Cracking Open the Black Box. MIT-IBM Seminar Series. Cambridge, MA. September 2020.

Human Agency and Network Rules: Rewriting a Generative Network. Google Magenta Group Meeting. Mountain View, CA. September 2020.

GAN Paint and GAN Rewriting. Boston University Computer Vision Semniar. Boston, MA. September 2020.

Interacting with the Structure of a Deep Net: Rewriting the Rules of a GAN. Adobe Research. San Jose, CA. August 2020.

Reflected Light and Doors in the Sky: Rewriting GANs. Advances in Image Manipulation Workshop, ETH Zurich. Zurich, Switzerland. August 2020.

Dissecting and Modifying the Rules Inside a GAN. Computer Vision Seminar, Berkeley. Berkeley, CA. August 2020.

Creativity, Human Agency and Rewriting Deep Generative Models. Computer Graphics Seminar, Stanford University. Palo Alto, CA. August 2020.

Semantic Photo Manipulation using a GAN. RealTime Conference at SIGGRAPH, June 2020.

Explaining the Units of Classifiers and Generators in Vision. Computer Vision Seminar, Brown University. Providence, RI. April 2020.

Dissecting the Semantic Structure of Deep Networks for Vision. Explainable AI for Vision Workshop. Seoul, Korea. November 2019.

Dissecting and Manipulating Generative Adversarial Networks. Image Synthesis Workshop. Seoul, Korea. October 2019.

Exploring a Generator with GANDissect. GANocracy Workshop. Cambridge, MA. May 2019.

Understanding the Internal Structure of a GAN. Re-Work Deep Learing Summit. Boston, MA. May 2019.

Dissecting Artificial Neural Networks for Vision. Martinos Center for Biomedical Imaging. Boston, MA. April 2019.

Semantic Paint using a Generative Adversarial Network. Samsung/MIT Design Workshop. Cambridge, MA. April 2019.

Dissecting What a Generative Network Can Learn Unsupervised. DARPA XAI PI Meeting. Berkeley, CA. February 2019.

Interpretation of Deep Networks for Vision. Trustworthy and Robust AI Initiative. Cambridge, MA. February 2019.

On the Units of Generative Adversarial Networks. AAAI Workshop on Network Interpretability. Honolulu, HI. January 2019.

Explaining Explanations: Interpretation of Deep Neural Networks. Trust.ML Workshop on Public Policy Aspects of ML. Cambridge, MA. June 2018.

David Bau, Ph.D.