Efficient Tactile Simulation with Differentiability for Robotic Manipulation

Jie Xu¹     Sangwoon Kim¹     Tao Chen¹     Alberto Rodriguez¹     Pulkit Agrawal¹     Wojciech Matusik¹     Shinjiro Sueda²

¹ Massachusetts Institute of Technology      ² Texas A&M University

Efficient simulation of tactile sensors can unlock new opportunities for learning tactile-based manipulation policies in simulation and then transferring the learned policy to real systems, but fast and reliable simulators for dense tactile normal and shear force fields are still under-explored. We present a novel approach for efficiently simulating both the normal and shear tactile force field covering the entire contact surface with an arbitrary tactile sensor spatial layout. Our simulator also provides analytical gradients of the tactile forces to accelerate policy learning. We conduct extensive simulation experiments to showcase our approach and demonstrate successful zero-shot sim-to-real transfer for a high-precision peg-insertion task with high-resolution vision-based GelSlim tactile sensors.