Real-time hand-tracking with a color glove

Robert Y. Wang and Jovan Popović

We demonstrate real-time tracking of the 3-D pose and configuration of the hand for gestural user-input and desktop virtual reality. The only components of our system are a cloth glove and a webcam.

Gestural user input

Tracking the 3-D pose and configuration of the hand

Our system is low latency (~100 ms) and can track the relatively fast motion of this sequence.

Desktop virtual reality

Our system can be used as an input device for desktop virtual reality.

Bimanual rotation and scaling

Here we demonstrate rotation and scaling of the Stanford bunny with both hands.

Manipulating a virtual yoke

Here we demonstrate controlling a (crude) flight simulator by manipulating a virtual yoke.

About the color glove

The color glove is printed on a Lycra fabric and does not contain any obstructive sensors. The pattern is designed to be distinctive and facilitate robust tracking.

Project and publication details

We're starting a company, 3Gear Systems, based in part on this technology.