In partial fullfillment of the Area Exam doctoral requirements:

Linear and Nonlinear Data Dimensionality Reduction

Abstract
This report discusses one paper for linear data dimensionality reduction, Eigenfaces, and two recently developed nonlinear techniques. The first nonlinear method, Locally Linear Embedding (LLE), maps the input data points to a single global coordinate system of lower dimension in a manner that preserves the relationships between neighboring points. The second method, Isomap, computes geodesic distances along a manifold as sequences of hops between neighboring points, and then applies Multidimensional Scaling (MDS) to these geodesic distances instead of Euclidean distances. To provide depth of understanding as well as background for comparison, the classical linear techniques MDS and Principle Component Analysis (PCA) are derived from three different approaches. The algorithmic, applicability, and implementation issues of the three papers are discussed in the common framework of data dimensionality reduction. Simple experimental results and suggestions for improvement are presented.

My Contribution

The Three Papers

1.)  "A Global Geometric Framework for Nonlinear Dimensionality Reduction"
J.B. Tenenbaum, V.d. Silva, J.C. Langford
Science, December 2000; 290:2319-2323.
2.)  "Nonlinear Dimensionality Reduction by Locally Linear Embedding"
S.T. Roweis, L.K. Saul
Science, December 2000; 290:2323-2326.
3.)  "Eigenfaces for Recognition"
M. Turk, A. Pentland
Journal of Cognitive Neuroscience, 1991; 3:71-86.

Useful Links