Research Projects

   Home Research Publications  CV  Teaching Students

Projects Publications SW CV

Computer Vision Student projects
Questions or Comments? Please email me.

Gaussian Process Latent Variable Models 3D Human Body Tracking


We use Scaled Gaussian Process Latent Variable Models (SGPLVM) to learn prior models of 3D human pose for 3D people
tracking. The SGPLVM simultaneously optimizes a low dimensional embedding of the high-dimensional pose data and a density function that both gives higher probability to points close to training data and provides a nonlinear probabilistic mapping from the low-dimensional latent space to the full-dimensional pose space. The SGPLVM is a natural choice when only small amounts of training data are available.
We demonstrate our approach with two distinct motions, golfing and walking.
We show that the SGPLVM sufficiently constrains the problem such that tracking can be accomplished with straighforward deterministic optimization.

Monocular 3D Tracking of the golf swing


We propose an approach to incorporating dynamic models into the human body tracking process that yields full 3--D reconstructions from monocular sequences.
We formulate the tracking problem in terms of minimizing a differentiable criterion whose differential structure is rich enough for successful optimization using a simple hill-climbing approach as opposed to a multi-hypotheses probabilistic one. In other words, we avoid the computational complexity of multi-hypotheses algorithms while obtaining excellent results under challenging conditions.

To demonstrate this, we focus on monocular tracking of a golf
swing from ordinary video. It involves both dealing with
potentially very different swing styles, recovering arm motions that are perpendicular to the camera plane and handling strong self-occlusions.

3D Human Body Tracking using Temporal Models


We use temporal motion models based on Principal Component Analysis (PCA) to formulate the tracking problem as one of minimizing differentiable objective functions. Our experiments show that the differential structure of these objective functions is rich enough to take advantage of standard deterministic optimization methods, whose computational requirements are much smaller than those of probabilistic ones and can nevertheless yield very good results even in difficult situations.

We show the effictiveness of our approach compare to probabilistic approaches, that while effective, require exponentially large amounts of computation as the number of degrees of freedom in the model increases.

To know more...

3D Tracking for Gait Characterization and Recognition

  Most current gait analysis algorithms rely on appearance-based methods that do not explicitly take into account the 3--D nature of the motion. In this work, we propose an approach that relies on robust 3--D tracking and has the potential to overcome the limitations of appearance-based approaches, such as their sensitivity to occlusions and changes in the direction of motion.

To know more...

Style-based Motion Generation


Representing motions as linear sums of principal components has become a widely accepted animation technique. While powerful, the simplest version of this approach is not particularly well suited to modeling the specific style of an individual whose motion had not yet been recorded when building the database: It would take an expert to adjust the PCA weights to obtain a motion style that is indistinguishable from his. Consequently, when realism is required, current practice is to perform a full motion capture session each time a new person must be considered.

We extend the PCA approach so that this requirement can be drastically reduced: For whole classes of motion such as walking or running, it is enough to observe the newcomer moving only once at a particular speed using either an optical motion capture system or a simple pair of synchronized video cameras. This one observation is used to compute a set of principal component weights that best approximates the motion and to extrapolate in real-time realistic animations of the same person walking or running at different speeds.

To know more...

Hierarchical Joint Limits


Our goal is to improve a joint limits representation that has the following characteristics:

1. captures intra- and inter-joint coupling,
2. is easy to derive on the basis of live motion recordings,
3. allows rapid determinant of the validity of a rotation.

Two main applications to a hierarchical modelization of a set of joint limits are shown: Video-based motion capture and animation

To know more...


 Useful Links

Prof. Trevor Darrell

International Computer Science Institute
1947 Center street, Suite 600 (room 513)
Berkeley, CA 94704

Phone: +1 510 666 2942
Email:, Fax: +1 510 666 2956


shareright © 2002 Phlash  

eXTReMe Tracker