Xavier Puig

Ph.D. Student, MIT
Email: xavierpuig@csail.mit.edu
Office: 32-385, 32 Vassar Street

ResumeGoogle ScholarLinkedInGitHub

About Me

I am a Ph.D. student at the Computer Science and Artificial Intelligence Laboratory (CSAIL) of MIT, advised by Professor Antonio Torralba. Before joining MIT, I obtained a double degree in Computer Science and Telecommunications at the CFIS program of UPC.

My research lies at the intersection of computer vision and machine learning. In particular, I study how to better understand on model complex human activities, and how to build agents that can help humans in performing them. I am also interested in topics in vision and language, neuro-symbolic AI, video understanding and representation learning.

I am also organizing the MIT Vision Seminar. Contact me if you would like to come present.

News

April 2021:
We are organizing the Social AI Virtual Gathering , at ICLR 2021.
February 2021:
We are organizing the Social Intelligence in Humans and Robots Workshop, to be held at ICRA'2021.
January 2021:
Watch-And-Help was accepted to ICLR'2021 as a Spotlight presentation.
November 2020:
Watch-And-Help was accepted to the Cooperative AI Workshop at NeurIPS 2020 as a Best paper award.
October 2020:
We released the code for our Watch-And-Help paper.
September 2020:
We released the new version of VirtualHome, as well as the Unity Source Code.

Publications

Generative Models as a Data Source for Multiview Representation Learning
Ali Jahanian, Xavier Puig, Yonglong Tian, Phillip Isola.
Paper Code
Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration
Xavier Puig, Tianmin Shu, Shuang Li, Ziling Wang, Josh Tenenbaum, Sanja Fidler, Antonio Torralba.
Cooperative AI Workshop at NeurIPS, 2020 - Best paper award
International Conference on Learning Representations (ICLR), 2021 - Spotlight
Paper Code Webpage
Synthesizing Environment-Aware Activities via Activity Sketches
Andrew Liao*, Xavier Puig*, Marko Boben, Antonio Torralba. Sanja Fidler,
In Proc. Computer Vision and Pattern Recognition (CVPR), 2019.
Paper Code Webpage
Semantic Understanding of Scenes through ADE20K Dataset
Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, Antonio Torralba.
International Journal on Computer Vision (IJCV), 2018.
Paper Dataset Benchmark Page Challenge Page Toolkit Demo
VirtualHome: Simulating Household Activities via Programs
Xavier Puig*, Kevin Ra*, Marko Boben*, Jiaman Li, Tingwu Wang, Sanja Fidler, Antonio Torralba,
In Proc. Computer Vision and Pattern Recognition (CVPR), 2018 - Oral
Paper Webpage Press Talk
Open Vocabulary Scene Parsing
Hang Zhao, Xavier Puig, Bolei Zhou, Sanja Fidler, Antonio Torralba.
In Proc. International Conference in Computer Vision (ICCV), 2017.
Paper Page
Scene Parsing through ADE20K Dataset
Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, Antonio Torralba.
In Proc. Computer Vision and Pattern Recognition (CVPR), 2017.
Paper Dataset Toolkit Demo

Experience

  • 2016.9 - present. Research Assistant, MIT Computer Science and Artificial Intelligence Laboratory
    Advised by Prof. Antonio Torralba
  • 2019.6 - 2019.8. Research Intern, Google Cambridge working in Contrastive Learning
    Advised by Dilip Krishnan, Aaron Maschinot, Aaron Sarna
  • 2017.6 - 2017.8. Research Intern, Google Seattle, working in Mobile Computer Vision and Adaptive Inference.
    Advised by Li Zhang, Yukun Zhu, Maxwell Collins

Professional Service