Re: Does Varimax always find maximum of variance in factor loadings?

Hi Reef -

again thanks for your msg. Well, things got clearer by that,
and there was a misunderstanding.

Yes - I know, entries should be +1 and -1. And the problem is
to find, which ones are the minus.

Now my idea was just that: it was described that Hadamard-matrices
have
- equal eigenvalues
- of value sqrt(n) .

Now, if any matrix is diagonizable, for instance the hadamard
matrix - then a diagonal matrix of exactly (well - numerical
precision out of question) eigenvalues of sqrt(n) should be
transformable to just the Hadamard-type matrix in question.

Now: how to find that back-rotation?
The diagonal-matrix has high variance in its squared column-
loadings. The Hadamard matrix has variance zero of that squared
loadings. Now - to find a solution just apply an iterative
process of rotation which is just designed to find that
zero-variance-position.
Well, I tried common methods before (it's not deep research,
I came across that problem in a recreational-math-discussion),
starting with the creation of orthogonal sequences of +1/-1
and such. Then the idea of using a completely canned and well
established method came to my mind.
Please note, that this *works* for n=4,n=8,n=12,16,24,32,48,64,
72,96 and I expect it works even for higher dimension - but the
cpu-consume is too high for my private PC to surpass that numbers.
What teased me, was simply, that there are configurations, of
far smaller size (like n=28 for instance), which do *not* work
(although the result is near with +-0.1 deviations. Again unfor-
tunately simply using the sign of the resulting values and fixing
then accordingly to +1/-1 is not sufficient)

So I assumed a special problem with that dimensions (n=4*7,4*15,...)

The finding of hadamard-matrices is obviously a problem which is
not solved completely - the sizes n=468 and n=628 (or something
like) were found just recently with work of monthes of cpu-cycles.
If there would be a way to make the simple vari-min-method work,
then this would be a nice and handy general method to overcome
that problem - at least principally.

On the other hand: in case, it could be shown, that vari-min
systematically cannot detect the zero-variance-solution in certain
cases, then this is possibly also a counterexample to the common
convergence-assumption of Varimax, too, and explains, *why* its
definite finding of extremas could not be proven until today.
(but again: for my part - that would only be a spinoff-result of
that special-case Hadamard-failings).



>
>
> If you minimize the variance of a set of non-negative (squares of
> the loadings) quantitities, without further stipulations, the
> minimum variance is ZERO, and that's when all the numbers are
> equal -- that's what I mean by CONSTANT == each number of each
> column must be the identical.
>
> That is of course a more general result than you need.
>
> In your case of a Hadamard matrix, whose definition is:
>
>
>>A Hadamard matrix is a square matrix with entiries +1, -1 whose
>>rows are mutually orthogonal.
>
>
> You already KNOW that the SQUARES of the loadings in the entire
> matrix are exactly 1, i.e., each column consists of all 1s, and
> the variance of each column is ZERO.

Ok, I hope I could clarify this misunderstanding:

Diagonal ---> Hadamard : what type of maximization to choose?
high variance in columns --> zero variance (of squared values) in columns
consequence: try varimin.

> If you minimize the variance of a set of non-negative (squares of
> the loadings) quantitities, without further stipulations, the
> minimum variance is ZERO, and that's when all the numbers are

That's what everyone would expect.
But varimin fails in some special cases. Why?

Regards -

Gottfried Helms
.

Relevant Pages

  • Re: nonlinear regression for dummies
    ... > variance, chi-squared, and so forth. ... But the output of the regression ... is easier and simpler, ordinary least squares regression. ... you would learn that the ANOVA model ordinarily ...
    (sci.stat.edu)
  • Re: VB6 math problem
    ... The standard deviation (statistics folk generally ... speak of "variance" of which s-d is the positive square root) ... But I'm still not quite sure why we need to measure the spread in terms of the sum of the squares of the difference of each of ... The sum of squares, and the need for calculating it, was very easy for me to understand in the electrical industry, where there was a definite and very clearly defined need to do it, but I cannot yet see why it is important in meauring the spread of data. ...
    (microsoft.public.vb.general.discussion)
  • Re: Large Standard Errors in Multinomial Logit Estimation
    ... Herman Rubin wrote... ... "In that case, the coefficients should be 0, except for the ... but the estimated variance should also be zero. ...
    (sci.stat.math)
  • Re: gaussian noise
    ... average of all means will be zero and var will be 1. ... You can only guarantee a particular mean and variance by generating ... variance(using randn in matlab) as .1 that var is changing from symbol ...
    (comp.dsp)
  • Re: why>?
    ... First, it's just variance, not ... this is equivalent to the average of the squares of the ... much faster to use spreadsheets than set up the tables needed to ... No, all you do it write reports, and all your little brain can ...
    (microsoft.public.excel)