 MPEG(65K) Quicktime(693K) |
Poggio and Brunelli first proposed the use of a set of 2D views of an object to generate novel, intermediate views, under the control of a set of parameters, such as the object's pose. Poggio and Brunelli also proposed the use of Hyper Basis Functions as the specific learning technique which learns the association between views and parameters, and is capable of generalizing to generate the novel intermediate views. Poggio and Brunelli also underscored the inherent non-smoothness of a map from high-level parameters to the pixel domain. Thus, they proposed that the hyper basis function should instead learn the mapping from high-level parameters to a set of control points in the views that are in correspondence with each other.
Librande developed a system called Xspace based on the earlier work by Poggio and Brunelli. The user first stores a set of images of an object into an example-base. A learning module, which consists of a Radial Basis Function, analyzes the example-base, and computes a "drawing function" which can generate new images. The learned map if from a set of user-defined parameters to a set of points on the example images. The points are defined and placed in correspondence using semi-automatic tools. Xspace has been deployed commercially.
Beymer,Shashua, Poggio showed that the learning techniques described above may be used for graphics synthesis of novel, grey-level (and color) images. A radial basis function was employed to learn the mapping from parameters to a set of example images of a face at different poses and expressions. The trained radial basis function would thus constitute a synthesis network, and thus, for any set of input parameters, a novel output image could be synthesized.
Beymer, Shashua, Poggio also showed that a radial basis function may also be used to learn the inverse mapping, from example images to the set of parameters. Thus, for any input novel image, the trained learning network could output a set of parameters indicating where in the parameter space the new image lay. The trained radial basis function thus constituted an analysis network.
Both analysis and synthesis networks can be combined to provide a novel approach of model-based compression, useful for video email and video teleconferencing. The following example illustrates the approach: an incoming image stream, shown on the left, is analyzed to yield parameters encoding head pose and smile information. These parameters are then fed into a trained synthesis network, which reconstructs the sequence, as shown on the right.
MPEG(65K) Quicktime(693K) |