This report discusses one paper for linear data dimensionality reduction, Eigenfaces, and two recently developed nonlinear techniques. The first nonlinear method, Locally Linear Embedding (LLE), maps the input data points to a single global coordinate system of lower dimension in a manner that preserves the relationships between neighboring points. The second method, Isomap, computes geodesic distances along a manifold as sequences of hops between neighboring points, and then applies Multidimensional Scaling (MDS) to these geodesic distances instead of Euclidean distances. To provide depth of understanding as well as background for comparison, the classical linear techniques MDS and Principle Component Analysis (PCA) are derived from three different approaches. The algorithmic, applicability, and implementation issues of the three papers are discussed in the common framework of data dimensionality reduction. Simple experimental results and suggestions for improvement are presented.
| 1.) |
"A Global Geometric Framework for Nonlinear Dimensionality Reduction"
J.B. Tenenbaum, V.d. Silva, J.C. Langford Science, December 2000; 290:2319-2323. |
| 2.) |
"Nonlinear Dimensionality Reduction by Locally Linear Embedding"
S.T. Roweis, L.K. Saul Science, December 2000; 290:2323-2326. |
| 3.) | "Eigenfaces for Recognition"
M. Turk, A. Pentland Journal of Cognitive Neuroscience, 1991; 3:71-86. |