Xavier Puig

Ph.D. Student, MIT
Email: xavierpuig@csail.mit.edu
Office: 32-385, 32 Vassar Street

ResumeGoogle ScholarLinkedInGitHub

About Me

I am a Ph.D. student at the Computer Science and Artificial Intelligence Laboratory (CSAIL) of MIT, advised by Professor Antonio Torralba. Before joining MIT, I obtained a double degree in Computer Science and Telecommunications at the CFIS program of UPC.

My research lies at the intersection of computer vision and machine learning. In particular, I study how to better understand on model complex human activities, and how to build agents that can help humans in performing them. I am also interested in the intersection of vision and language, neuro-symbolic AI, semantic segmentation, video understanding and representation learning.

I am also organizing the MIT Vision Seminar. Contact me if you would like to come present.


April 2021:
We are organizing the Social AI Virtual Gathering , at ICLR 2021.
February 2021:
We are organizing the Social Intelligence in Humans and Robots Workshop, to be held at ICRA'2021.
January 2021:
Watch-And-Help was accepted to ICLR'2021 as a Spotlight presentation.
November 2020:
Watch-And-Help was accepted to the Cooperative AI Workshop at NeurIPS 2020 as a Best paper award.
October 2020:
We released the code for our Watch-And-Help paper.
September 2020:
We released the new version of VirtualHome, as well as the Unity Source Code.


Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration
Xavier Puig, Tianmin Shu, Shuang Li, Ziling Wang, Josh Tenenbaum, Sanja Fidler, Antonio Torralba.
Cooperative AI Workshop at NeurIPS, 2020 - Best paper award
International Conference on Learning Representations (ICLR), 2021 - Spotlight
Paper Code
Synthesizing Environment-Aware Activities via Activity Sketches
Andrew Liao*, Xavier Puig*, Marko Boben, Antonio Torralba. Sanja Fidler,
In Proc. Computer Vision and Pattern Recognition (CVPR), 2019.
Paper Code Webpage
Semantic Understanding of Scenes through ADE20K Dataset
Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, Antonio Torralba.
International Journal on Computer Vision (IJCV), 2018.
Paper Dataset Benchmark Page Challenge Page Toolkit Demo
VirtualHome: Simulating Household Activities via Programs
Xavier Puig*, Kevin Ra*, Marko Boben*, Jiaman Li, Tingwu Wang, Sanja Fidler, Antonio Torralba,
In Proc. Computer Vision and Pattern Recognition (CVPR), 2018 - Oral
Paper Webpage Press Talk
Open Vocabulary Scene Parsing
Hang Zhao, Xavier Puig, Bolei Zhou, Sanja Fidler, Antonio Torralba.
In Proc. International Conference in Computer Vision (ICCV), 2017.
Paper Page
Scene Parsing through ADE20K Dataset
Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, Antonio Torralba.
In Proc. Computer Vision and Pattern Recognition (CVPR), 2017.
Paper Dataset Toolkit Demo

Professional activities

  • Conference reviewer for CVPR'19, CVPR'20, ECCV'19, ICCV'20
  • Research Intern at Google Seattle (2017), working in Mobile Vision with Li Zhang
  • Research Intern at Google Cambridge (2019), working in Contrastive Learning with Dilip Krishnan, Aaron Maschinot, Aaron Sarna
  • Co-chair of the MIT Vision Seminar (2018-2019) and Chair (2019-Present)