In this work, we describe image-based synthesis techniques that make possible the creation of computer models of real human faces. The computer model is built using example images of the face, bypassing the need for any three-dimensional models. Intermediate views are synthesized through the use of a trained learning network which associates each of the example images with a set of pose and expression parameters. Control of the model is achieved by using the trained network to synthesize a new image of the face for any desired setting of pose and expression parameters.
We will also describe analysis techniques to ``analyze'' novel image streams of the same human face using the trained synthesis networks. For each incoming novel frame, these analysis techniques estimate the set of parameters that best match the model. The analysis algorithms are robust to moderate changes in translation, onplane rotation, scale, lighting changes, background changes, and even radical hairstyle changes!
Together, analysis and synthesis networks can serve as a basis for model-based compression that may be useful for video email and video teleconferencing. Other applications include human-driven animation (analysis alone) and synthetic human characters and avatars (synthesis alone).