I am a final-year Ph.D. student at MIT CSAIL, advised by Professor Wojciech Matusik in the Computational Design and Fabrication Group (CDFG). I obtained my bachelor's degree from Department of Computer Science and Technology at Tsinghua University with honor in 2016. During my undergraduate period, I worked with Professor Shi-Min Hu in the Tsinghua Graphics & Geometric Computing Group and with Professor Stelian Coros as an exchange student at CMU.

My research mainly focuses on the intersection of Robotics, Simulation, and Machine Learning. Specifically, I am interested in the following topics: Robotics Control, Reinforcement Learning, Differentiable Physics-based Simulation, Robotics Control and Design Co-Optimization, and Sim-to-Real.

Jie Xu

I am a final-year Ph.D. student at MIT CSAIL, advised by Professor Wojciech Matusik in the Computational Design and Fabrication Group (CDFG). I obtained my bachelor's degree from Department of Computer Science and Technology at Tsinghua University with honor in 2016. During my undergraduate period, I worked with Professor Shi-Min Hu in the Tsinghua Graphics & Geometric Computing Group and with Professor Stelian Coros as an exchange student at CMU.

My research mainly focuses on the intersection of Robotics, Simulation, and Machine Learning. Specifically, I am interested in the following topics: Robotics Control, Reinforcement Learning, Differentiable Physics-based Simulation, Robotics Control and Design Co-Optimization, and Sim-to-Real.

News

06/2022: Our work about UAV grammar for UAV co-design is accepted by IROS 2022.
05/2022: Our tactile manipulator design pipeline is covered by MIT News.
03/2022: A paper is conditionally accepted by SIGGRAPH 2022 (journal track).
02/2022: DiffCloth is accepted by TOG.
01/2022: Two papers are accepeted by ICRA 2022.
01/2022: Our work on using differentiable simulation for policy learning on complex control tasks is accepted by ICLR 2022.
12/2021: EvoGym is covered by MIT News.
11/2021: Our work on challenging in-hand re-orientation wins Best Paper Award at CoRL 2021.
09/2021: Our work EvoGym, a large-scale co-design benchmark is accepted by NeurIPS 2021.
09/2021: Our work on challenging in-hand manipulation is accepeted as an oral paper (6.5%) by CoRL 2021.
07/2021: DiffHand is covered by MIT News.
06/2021: Join NVIDIA simulation team as a research intern.
05/2021: Our work about differentiable simulation for robot co-design is accepted by RSS 2021.
03/2021: A paper on multi-objective robot topology search is accepted by ICRA 2021.
11/2020: RoboGrammar is covered by MIT News.

Publications

Accelerated Policy Learning with Parallel Differentiable Simulation
Jie Xu, Viktor Makoviychuk, Yashraj Narang, Fabio Ramos, Wojciech Matusik, Animesh Garg, Miles Macklin
International Conference on Learning Representations (ICLR 2022)
[Project Page] | [Paper] | [Video] | [Code is coming soon]

Closed-Loop Control of Direct Ink Writing via Reinforcement Learning
Michal Piovarči*, Michael Foshey*, Jie Xu, Timothy Erps, Vahid Babaei, Piotr Didyk, Szymon Rusinkiewicz, Wojciech Matusik, Bernd Bickel
ACM Transactions on Graphics (Proc. SIGGRAPH 2022)
[Project Page] | [Paper] | [Supp]

An Integrated Design Pipeline for Tactile Sensing Robotic Manipulators
Lara Zlokapa, Yiyue Luo, Jie Xu, Michael Foshey, Kui Wu, Pulkit Agrawal, Wojciech Matusik
IEEE International Conference on Robotics and Automation (ICRA 2022)
[Project Page] | [Code] | [MIT News]

Graph Grammar-Based Automatic Design for Heterogeneous Fleets of Underwater Robots
Allan Zhao, Jie Xu, Juan Salazar, Wei Wang, Pingchuan Ma, Daniela Rus, Wojciech Matusik
IEEE International Conference on Robotics and Automation (ICRA 2022)
[Paper]

DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact
Yifei Li, Tao Du, Kui Wu, Jie Xu, Wojciech Matusik
ACM Transactions on Graphics 2022
[Project Page] | [Arxiv]

An End-to-End Differentiable Framework for Contact-Aware Robot Design
Jie Xu, Tao Chen, Lara Zlokapa, Michael Foshey, Wojciech Matusik, Shinjiro Sueda, Pulkit Agrawal
Robotics: Science and Systems (RSS 2021)
[Project Page] | [Paper] | [Supp] | [Sim Doc] | [Code] | [Video] | [Poster] | [Talk] | [MIT News]

A System for General In-Hand Object Re-Orientation
Tao Chen, Jie Xu, Pulkit Agrawal
Conference on Robot Learning (CoRL 2021) (Best Paper Award) (Oral: 6.5%)
[Project Page] | [Paper] | [Supp] | [Arxiv] | [Video] | [BibTeX] | [MIT News]

Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robots
Jagdeep Singh Bhatia, Holly Jackson, Yunsheng Tian, Jie Xu, Wojciech Matusik
Conference on Neural Information Processing Systems (NeurIPS 2021)
[Project Page] | [Paper] | [Supp] | [Code] | [Video] | [BibTeX] | [MIT News]

Multi-Objective Graph Heuristic Search for Terrestrial Robot Design
Jie Xu, Andrew Spielberg, Allan Zhao, Daniela Rus, Wojciech Matusik
IEEE International Conference on Robotics and Automation (ICRA 2021)
[Project Page] | [Paper] | [Arxiv] | [Video] | [Talk]

Prediction-Guided Multi-Objective Reinforcement Learning for Continuous Robot Control
Jie Xu, Yunsheng Tian, Pingchuan Ma, Daniela Rus, Shinjiro Sueda, Wojciech Matusik
International Conference on Machine Learning (ICML 2020)
[Project Page] | [Paper] | [Supp] | [Code] | [Video] | [Talk]

RoboGrammar: Graph Grammar for Terrain-Optimized Robot Design
Allan Zhao, Jie Xu, Mina Konaković Luković, Josephine Hughes, Andrew Spielberg, Daniela Rus, Wojciech Matusik
ACM Transactions on Graphics 39(6) (Proc. SIGGRAPH Asia 2020)
[Project Page] | [Paper] | [Video] | [Code] | [MIT News]

Learning to Fly: Computational Controller Design for Hybrid UAVs with Reinforcement Learning
Jie Xu, Tao Du, Michael Foshey, Beichen Li, Bo Zhu, Adriana Schulz, Wojciech Matusik
ACM Transactions on Graphics 38(4) (Proc. SIGGRAPH 2019)
[Project Page] | [Paper] | [Video] | [Code] | [MIT CSAIL News]

Interactive Design Space Exploration and Optimization for CAD Models
Adriana Schulz, Jie Xu, Bo Zhu, Changxi Zheng, Eitan Grinspun, Wojciech Matusik
ACM Transactions on Graphics 36(4) (Proc. SIGGRAPH 2017)
[Project Page] | [Paper] | [Supplementary] | [Code] | [Video]

View Suggestion for Interactive Segmentation of Indoor Scenes
Sheng Yang, Jie Xu, Kang Chen, Hong-Bo Fu
Computational Visual Media, Vol. 3, Issue 2, pp 131-146, June 2017
[Paper] | [Video] | [Supplementary]

Extracting Sharp Features from RGB-D Images
Yan-Pei Cao, Tao Ju, Jie Xu, Shi-Min Hu
Computer Graphics Forum, Vol. 35, No. 8, pp. 138-174, 2017
[Paper]

Thesis

Learning to Fly: Computational Controller Design for Hybrid UAVs with Reinforcement Learning
Advisor: Wojciech Matusik
Master Thesis, Massachusetts Institute of Technology

Interactive Point Cloud Segmentation (in Chinese)
Advisors: Song-Hai Zhang, Shi-Min Hu
Undergraduate Thesis, Tsinghua University

Professional Experience

Research Intern at NVIDIA
Mentor: Miles Macklin

June 2021 - Aug 2021

Research Intern at Snap Research
Mentors: Guru Krishnan, Shree Nayar

June 2020 - Aug 2020

Awards & Honors

Gold Medal in the ACM-ICPC World Finals 2015
by Association for Computing Machinery

May 2015

China Computer Federation Outstanding Undergradate Award
by China Computer Federation

Oct 2015

Outstanding Graduate in Tsinghua University
by Tsinghua University

June 2016

Miscellaneous

MIT Table Tennis Team Member

Sept 2018 - present

Championship in 2018 MIT Open U1700 Division (USATT rated)
Singles

Oct 2018

Championship in NCTTA Upper New England Division in season 2018-2019
MIT Table Tennis Co-ed Team

Feb 2019

Third place in NCTTA Upper New England Division Men's Singles
Singles

Feb 2022