We propose an approach to automatically reconstruct entire 3D objects from a single 2D image, by using prior 3D shape models of classes (the 3D Potemkin model). The prior 3D shape models, defined as a collection of oriented primitive shapes centered at fixed 3D positions, can be learned from a few labeled images for each class. This class model can then be used to estimate 3D shape of an object instance, including occluded parts, from a single image. The reconstructed 3D shape is sufficiently accurate for a robot to estimate the pose of an object and successfully grasp it, even in situations where the part to be grasped is not visible in the input image.
The manipulation process is an open-loop
grasp (the mechanical action is only based on the initial command) without using
any touch sensors or feedbacks from the robot during the grasp process. All we
have is one single 2D image (no stereo or range data) and one known 3D ground
plane.
Demos
Robot
Manipulation
Publications
Han-Pang Chiu, Huan Liu, Leslie Pack Kaelbling, and Tomas Lozano-Perez,
"Class-Specific Grasping of 3D Objects from a Single 2D Image", IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS), 2010.
pdf
Han-Pang Chiu, Leslie Pack Kaelbling, and Tomas Lozano-Perez, "Automatic Class-Specific 3D Reconstruction from a Single Image", MIT CSAIL Technical Report MIT-CSAIL-TR-2009-008, February 2009. pdf