Re: Principal Axes of Orthogonal Column Vector set?

From: Gottfried Helms (helms_at_uni-kassel.de)
Date: 08/15/04 

Date: Sun, 15 Aug 2004 21:40:19 +0200

Am 15.08.04 16:42 schrieb eleaticus:

> "Gottfried Helms" <helms@uni-kassel.de> wrote in message
> news:cfn2rl$9th$00$1@news.t-online.com...
>
>
>>or is it that you need
>>understanding, how to actually find that rotation?
>
>
> viele danke.
>
> Yes, I needed to find that rotation.
>
>
>> But I don't know yet, whether ;-) the responses helped for the original
>>question?
>
>
> I think so. I have no direct knowledge of the correctness of the process
> cited but Herr Koopman's response seems to be complete to the level of
> detail I need. If you agree with it I have all I hoped for.
>
>
>>Was your focus the problem with an eventually false statement
>>in the books (maybe you only misread the passage)
>
>
> I had tried a straight eigenvalue/eigenvector solution of my triangle matrix
> and dead-ended. Apparently (basing this on Koopman's prescription) not
> without reason.
>
The jacobi-priciple is iterative; you just have to find the best
rotation in one pair of axes, then in another... so with all pairs
of axes. One of this pass rotates your matrix in a better position
as before. You have to repeat the whole process until a certain
numerical precision. The theoretical limit is the eigenvector-solution.

This jacobi-method is known as numerically extremely stable (don't
know why actually, but there is much material about research online)

I like most that type of a mental reminder, a geometrical view:
(start with a two-axes, two-dimensional sketch x,y,
 -sorry, no graphics here in ASCII):

If you search the centroid, this is just the mean-vector: you add
all vector-coordinates xi,yi into xm and ym, divide by n (the number
of vectors) and get the centroid-vector.
For multiple dimensions this is as easy as before; just do it in all
dimensions(axes) simultaneously.
This finds the center, useful for instance if all vectors are pointing
to equal masses.

But for some questions, that is not the best path. Assume, you would
be interested in finding that axis, which explains best the overall
direction of an assumed elliptical formed arrangement of the vectors.
Then, if many vectors point in opposite directions they cancel
each other and as the main direction a single, but conceptually
nonrelevant vector could dominate the whole system. So you might enter
the idea to just use the absolute coordinates, or better: to
mirror opposing vectors... but that's not good mathematics, since
this process cannot be set up in continuous formulae.

The search of principal axes means just this question.
A better idea than using the centroid and mirror the vectors, is,
to double the angle of each vector, so that eventually opposing
vectors overlay each other in the same direction. So they don't cancel
each other but add their weight - and it is a analytically continuous
operation.
If you find the center of these new vectors, you just use the
centroid method. After you found the center of the doubled angles,
you must halve its centroids' angle to find the vector, which
represents the principal axis.

(Equivalently you can do to find the quartimax or even varimax
method by multiplying the angles by four, thus enhancing the
centroid w.r. to two axes irrespectively of their direction
in one or two dimensions - just do a graphical sketch to see,
what I mean.)

The formula for doubling angles of a vector is

  xi' = xi*xi - yi*yi yi' = 2*xi*yi

Now you can select an option, whether the length of
a vector should be relevant or not. In your case it looks,
it is not relevant. So you divide both new coordinates by
the length of the vector.

Then you find the centroid of all new coordinates by
just summing them, getting
  xm' = x1'+x2'+... ym' = y1'+y2'+...

Now you have to halve the angle of the found centroid and
have the vector pointing to the principal axis.

--------------

Algebraically that is the same formula that you would get,
if you do calculus to maximize the variances in xi and minimize
that in yi, which is the definition of principal axes.
That is also identical to the eigenvector-problem: the x and
y-column are orthogonal.

Fortunately this process, done for all pairs of axes, and
iterated, converges well (even for the quartimax/varimax-
problem)

I saw the jacobi method written as an infinite matrix-
multiplication of the original matrix M with each
two-dimensional rotationmatrix Tj

  P = M*T1*T2*T3*T4*...

where the Tj-matrices cannot be determined beforehand, but
must computed at each step: if on iteration j

  C = Mj'*Mj

then the four entries in T different from 1 holding the
cos and sin for the rotation in two axes can be computed
from any off-diagonal-element of C with the goal to make
it zero. it can be shown, that on each step Cj gets more
diagonal. (The householder and other more refined methods
compute more entries for Tj in a single step, thus are
more efficient).

The result is that

  Coo --> diagonal
  and T = T1*T2*T3*.... is the rotation-matrix, that brings
                        M into its principal axes position P.

 Norming the columns xi, yi,... in P to have the norm 1
 into a matrix E exhibits the eigenvectors of M in E.

-------------------

> My knowledge-based problem is currently my lack of resources and the residue
> of years gone by:
>
> a. my reading of the principal axes solution told me it was/should-be based
> on the description of vectors on an orthogonal basis.

Yes.

>
> b. factor analysis in the 'principal factors' sense was based on the
> description of vectors on a non-orthogonal basis.

It's just an improvement. The first goal of factor analysis was to
find one factor at all (the "intelligence" factor). Later the method
was improved to find more factors simultaneous. Primarily they were
thought as orthogonal, and this is likely the first shot in starting
the discussion in many new empirical surveys. It was also much more diffi-
cult to find non-orthogonal ("oblique") factors analytically instead
of orthogonal factors without the help with computers.
But since we *have* computers now, and we merely deal with conceptual
dimensions which could be correlated instead of abstract, strict
orthogonal factors, the oblique methods became more standard, and
needed a new label. Maybe they selected "factors" for the more
conceptual approach, where a strict orthogonality was not required.
and reserved the word "axes" then for the stric orthogonal, maybe more
abstract, approach. Today I favor the expression "latent variable"
even over "oblique factors" - but that's another story.

>
> c. I did not see anything that suggested to me there was a post factoring
> correction for b.

???? (Sorry... maybe it just a language-problem.)

>
> Hence, my uncertainty and curiousity as to whether or not the 'maximum
> variance' rotation of 'principal factors' were the actual principal axes.
>

Hmm. I never worked out the mathematics of oblique rotation. But
it seems very likely to me, that the concept of "maximizing
variance" on the oblique axes provides higher values in an
oblique system than in an orthogonal. so that concept may
leave that eigenvalue/eigenvector-technique for improving the
"variance-maximizing" approach.

> Hence, my wish to test the obvious theses directly.

Btw, just curious: with what software are you doing your testing?
Maybe, my selfmade "Matmate" could help to make things transparent?

Gottfried Helms

Relevant Pages

  • Re: A Matlab question on Patch
    ... axesR) where 2 real-head models get loaded. ... patch in some direction, please. ... human brain) into one axes of my figure. ... here is a function i wrote for a 3-D rotation ...
    (comp.soft-sys.matlab)
  • Re: 3D mouse rotation
    ... > axis. ... After including a visual representation of the camera axes on-screen, ... rotation, there is no residual rotation around the Z ...
    (comp.graphics.api.opengl)
  • Re: describing angular velocity in three dimensions around a single axis
    ... Imagine a body with an orthogonal axes set at it's centre of mass. ... as a single angular velocity around a seperate axis going through the ... I understand that the orientations of this new axis will be ... a finite time-step aka Euler's method of describing any rotation in 3D ...
    (sci.physics)
  • Re: Need to make some kind of coordinate transformation to 3D
    ... A rotation matrix has to be orthogonal with determinant=1. ... orthogonality, so I don't make this mistake again in the future? ... I guess something with dot product or cross product of the columns ...
    (comp.soft-sys.matlab)
  • Re: Galileo transformation
    ... |> A is the matrix for rotation and/or translation of the coordinate ... |> Orthogonality refers to a *relationship* between two things. ... that put both the rotation and the translation in a single 4x4 ...
    (sci.physics)