Real-Time Hand-Tracking with a Color Glove


Robert Y. Wang and Jovan Popović

We describe a system that can reconstruct the pose of the hand from a single image of the hand wearing a multi-colored glove. We demonstrate our system as a user-input device for desktop virtual reality applications.


Articulated hand-tracking systems have been widely used in virtual reality but are rarely deployed in consumer applications due to their price and complexity. In this paper, we propose an easy-to-use and inexpensive system that facilitates 3-D articulated user-input using the hands. Our approach uses a single camera to track a hand wearing an ordinary cloth glove that is imprinted with a custom pattern. The pattern is designed to simplify the pose estimation problem, allowing us to employ a nearest-neighbor approach to track hands at interactive rates. We describe several proof-of-concept applications enabled by our system that we hope will provide a foundation for new interactions in modeling, animation control and augmented reality.

New! Gestural user input and hand tracking videos

Older Results / Videos

Offline sequences

Rigid motion
(QuickTime) (WMV)

Free jiggle
(QuickTime) (WMV)

(QuickTime) (WMV)

Real-time demos

ASL finger spelling
(QuickTime) (WMV)

Finger walking
(QuickTime) (WMV)


Robert Y. Wang and Jovan Popovic, Real-Time Hand-Tracking with a Color Glove, ACM Transaction on Graphics (SIGGRAPH 2009), 28(3), August 2009.

    title={{Real-time hand-tracking with a color glove}},
    author={Robert Y. Wang and Jovan Popovi{\'c}},
    journal={ACM Transactions on Graphics},

Fast forward



We thank Sylvain Paris, Frédo Durand, and Randall Davis for their regular feedback; and Anthony Liu for his help with the real-time implementation. This work was supported by grants from the Foxconn Technology Group and software donations from Autodesk and Adobe Systems.