Re: Principal Component Analysis.

I've developed a program for PCA and classification
occurs well.. I've
used a library for finding eigen values and vectors
but little
understand how they work. why do the eigen vectors of
a covariance
matrix (NXN for N dimensions) give the direction of
maximum variance?

Would appreciate pointers to introductory material on
eigen values and
vectors. Googled but everything looks so complex,
sort of difficult to
comprehend as its the first time i'm doing something
like this.

Thanks

Joel



Suppose x is N-dimensional and has covariance matrix Sigma.

The first principal component is y1 = a1'x, where a1'a1 = 1 (that is, the sum of squares of the coefficients is one) which maximizes the variance of y1. The matrix algebra along with LaGrange multipliers, lead to a1 being the eigenvector corresponding to the largest eigenvalue.

Then sequentially, each yj being a linear combination of x and uncorrelated with the previous y1, ..., y(j-1), results in the other eigenvectors as coefficients, corresponding to the ordered eigenvalues.

You can also think of this process geometrically as a rotation of the coordinate axes in the direction of the principal axes.

I don't know if this helps.

Jack
.