I am a Research Scientist at NVIDIA Seattle Robotics Lab led by Professor Dieter Fox. I obtained my Ph.D. degree in Computer Science at MIT CSAIL in 2022 under the supervision of Professor Wojciech Matusik in the Computational Design and Fabrication Group (CDFG). I obtained my bachelor's degree from Department of Computer Science and Technology at Tsinghua University with honor in 2016. During my undergraduate period, I worked with Professor Shi-Min Hu in the Tsinghua Graphics & Geometric Computing Group and with Professor Stelian Coros as an exchange student at CMU.

My research mainly focuses on the intersection of Robotics, Simulation, and Machine Learning. Specifically, I am interested in the following topics: Robotics Control, Reinforcement Learning, Differentiable Physics-based Simulation, Robotics Control and Design Co-Optimization, and Sim-to-Real.

Jie Xu

I am a Research Scientist at NVIDIA Seattle Robotics Lab led by Professor Dieter Fox. I obtained my Ph.D. degree in Computer Science at MIT CSAIL in 2022 under the supervision of Professor Wojciech Matusik in the Computational Design and Fabrication Group (CDFG). I obtained my bachelor's degree from Department of Computer Science and Technology at Tsinghua University with honor in 2016. During my undergraduate period, I worked with Professor Shi-Min Hu in the Tsinghua Graphics & Geometric Computing Group and with Professor Stelian Coros as an exchange student at CMU.

My research mainly focuses on the intersection of Robotics, Simulation, and Machine Learning. Specifically, I am interested in the following topics: Robotics Control, Reinforcement Learning, Differentiable Physics-based Simulation, Robotics Control and Design Co-Optimization, and Sim-to-Real.


News

05/2023: One paper accepted to ICML 2024 and two papers accepted to RSS 2024.
08/2023: One paper accepted to CoRL 2023 (Oral).
01/2023: We release the code for our Tactile Simulation Paper (CoRL 2022).
10/2022: SHAC is covered by Two Minute Papers.
10/2022: I started as a Research Scientist at Nvidia Seattle Robotics Lab.
09/2022: Our work on differentiable tactile simulation has been accepted by CoRL 2022.
08/2022: I successfully defended my Ph.D. thesis and obtained the Ph.D. degree.
08/2022: Our work on general assembly path planning has been conditionally accepted by SIGGRAPH Asia 2022 (Journal Track).
08/2022: Our work about 3d printing control by reinforcement learning is covered by MIT News.
07/2022: We officially release the code of SHAC, please check it out.
06/2022: Our work about UAV grammar for UAV co-design is accepted by IROS 2022.
05/2022: Our tactile manipulator design pipeline is covered by MIT News.
03/2022: A paper is conditionally accepted by SIGGRAPH 2022 (journal track).
02/2022: DiffCloth is accepted by TOG.
01/2022: Two papers are accepeted by ICRA 2022.
01/2022: Our work on using differentiable simulation for policy learning on complex control tasks is accepted by ICLR 2022.
12/2021: EvoGym is covered by MIT News.
11/2021: Our work on challenging in-hand re-orientation wins Best Paper Award at CoRL 2021.
09/2021: Our work EvoGym, a large-scale co-design benchmark is accepted by NeurIPS 2021.
09/2021: Our work on challenging in-hand manipulation is accepeted as an oral paper (6.5%) by CoRL 2021.
07/2021: DiffHand is covered by MIT News.
06/2021: Join NVIDIA simulation team as a research intern.
05/2021: Our work about differentiable simulation for robot co-design is accepted by RSS 2021.
03/2021: A paper on multi-objective robot topology search is accepted by ICRA 2021.
11/2020: RoboGrammar is covered by MIT News.


Publications

MeMo: Meaningful, Modular Controllers via Noise Injection
Megan Tjandrasuwita, Jie Xu, Armando Solar-Lezama, Wojciech Matusik
Conference on Neural Information Processing Systems (NeurIPS 2024)
[Paper] | [Code]

Graph Grammar-Based Automatic Design for Heterogeneous Fleets of Underwater Robots
Allan Zhao, Jie Xu, Juan Salazar, Wei Wang, Pingchuan Ma, Daniela Rus, Wojciech Matusik
IEEE International Conference on Robotics and Automation (ICRA 2022)
[Paper]

A System for General In-Hand Object Re-Orientation
Tao Chen, Jie Xu, Pulkit Agrawal
Conference on Robot Learning (CoRL 2021) (Best Paper Award) (Oral: 6.5%)
[Project Page] | [Paper] | [Supp] | [Arxiv] | [Video] | [BibTeX] | [MIT News]

Multi-Objective Graph Heuristic Search for Terrestrial Robot Design
Jie Xu, Andrew Spielberg, Allan Zhao, Daniela Rus, Wojciech Matusik
IEEE International Conference on Robotics and Automation (ICRA 2021)
[Project Page] | [Paper] | [Arxiv] | [Video] | [Talk]

View Suggestion for Interactive Segmentation of Indoor Scenes
Sheng Yang, Jie Xu, Kang Chen, Hong-Bo Fu
Computational Visual Media, Vol. 3, Issue 2, pp 131-146, June 2017
[Paper] | [Video] | [Supplementary]

Extracting Sharp Features from RGB-D Images
Yan-Pei Cao, Tao Ju, Jie Xu, Shi-Min Hu
Computer Graphics Forum, Vol. 35, No. 8, pp. 138-174, 2017
[Paper]


Thesis

Towards Computational Design of Shape and Control for Rigid Robots
Thesis Advisor: Wojciech Matusik
Committee: Daniela Rus, Pulkit Agrawal
Ph.D. Thesis, Massachusetts Institute of Technology
[Thesis] | [Video]



Interactive Point Cloud Segmentation (in Chinese)
Advisors: Song-Hai Zhang, Shi-Min Hu
Undergraduate Thesis, Tsinghua University


Professional Experience

Research Intern at NVIDIA
Mentor: Miles Macklin

June 2021 - Aug 2021

Research Intern at Snap Research
Mentors: Guru Krishnan, Shree Nayar

June 2020 - Aug 2020

Awards & Honors

Best Paper Awards at CoRL 2021

Nov 2021

Gold Medal in the ACM-ICPC World Finals 2015
by Association for Computing Machinery

May 2015

China Computer Federation Outstanding Undergradate Award
by China Computer Federation

Oct 2015

Outstanding Graduate in Tsinghua University
by Tsinghua University

June 2016

Miscellaneous

Sept 2018 - Aug 2022

Championship in 2018 MIT Open U1700 Division (USATT rated)
Singles

Oct 2018

Championship in NCTTA Upper New England Division in season 2018-2019
MIT Table Tennis Co-ed Team

Feb 2019

Third place in NCTTA Upper New England Division Men's Singles
Singles

Feb 2022