Analysis refers to the extraction, from a novel image sequence, of parameters encoding such high-level information as an object's position, orientation, and deformation. In this work, we employ an analysis-by-synthesis approach to analyze facial images. The cornerstones of the approach are as follows:

• Synthesis networks of the face are first constructed using examples of the face. Each network is parametrized according to whatever particular facial motions the user wants to capture from the novel incoming image sequences.
• The synthesis network chosen is then embedded into the analysis routines to find and track the face in the incoming image sequence.
• The analysis routines can extract both flexible parameters, such as eye gaze and mouth orientation, as well as affine parameters, such as head position and scale. In general, the analysis routines are robust to moderate changes in lighting conditions and hairstyle changes.

Figure A MPEG(28K) Quicktime(292K)

Figure B MPEG(30K) Quicktime(390K)

Figures A and B above depict two sets of analysis test results. In figure A, a 9-example synthesis network that encoded left-to-right and up-to-down head pose movement was used to analyze a novel sequence in which the head moved to the bottom and then to the left. In figure B, a 14-example network encoding pose, eye, and mouth movement is used to analyze a novel sequence involving mouth movement alone. The MPEG's and Quicktime's shown depict both the original and the synthesized sequences side by side, for comparison.

For a more complete set of test results depicting analysis on a variety of facial movements, please look here.