Re: Difference between Principal Components Analysis and Factor Analysis?

On 26 May 2006 00:38:32 -0700, "2046" <2046AB@xxxxxxxxx> wrote:

Hi,

What is the difference between PCA and Factor Analysis?

If you base them on correlations -
PCA analyzes a matrix with 1.0s on the main diagonal,
whereas FA analyzes the matrix with estimates
of the "communalities".

PCA is a transformation of the original axes which preserves
all the original variance, and results in an orthogonal
representation of the axes. "Orthogonal" can be useful;
sometimes only the largest roots are used, which is similar
to the use of FA.

By using communalities, usually estimated iteratively,
the FA attempts to ignore the 'unique' variance of the
separate items while partitioning their 'common' variance.
There are presumed to be a few latent factors which
represent whatever was measured by many items.

FA is universally used in scale development. Every time I
receive data with a set of Likert-type items or yes/no items
(and a useful N), I check to see whether the correlations are
reasonably large, and if the "items" load where their labels
suggest that they should. The 'factors' are usually a useful
way to reduce the number of statistical tests to be performed
later on, replacing several dozen items with a few Factors of
higher reliability than a single item.

Even if there are a-priori Factors because the scale is standard
and well known -- and no one intends to present a new
factoring -- this check is a way to detect problems that occurred
in the data collection; or it may provide evidence of essential
differences between the Norming samples and the sample on hand.


Roderick McDonald wrote a nice book on "Factor analysis and
related methods" which was easy to read. He also wrote a newer
book, "Test theory: a unified approach."


I read some sneering in a couple of other notes --
I think they are objecting to presentations of a by-gone day.

In the 1930s, there was some optimism that PCA or FA might
hold a key to understanding "intelligence" or what-not -- Is there
one specific "g" for ability, or are there separate talents? --
but it eventually became evident that rotations are always rather
arbitrary. And then there is the choice of Sample, and how that
affects the correlations and structure in drastic ways. I think the
consensus emerged that it is fairly useless to argue between
rotation methods, or between PCA and FA, if you are looking
for one method to support one ontological truth.


Factoring was too tedious for most people to do until computers
made fast solutions available, mainly in the 1970s. I used to
hear that educators still wanted to interpret Factors as strong
evidence for meaning, but I haven't seen any of that in a long
time.


--
Rich Ulrich, wpilib@xxxxxxxx
http://www.pitt.edu/~wpilib/index.html
.

Relevant Pages

  • Re: Principal Component Analysis
    ... If your categorical variables are ordered-categorical, ... Pearson correlations between each pair of continuous variables. ... then place these in a single matrix and analyze that matrix by PCA. ... hierarchical cluster analyis on the correlation matrix. ...
    (sci.stat.consult)
  • Re: IPCC rules Forest Gate shooting accidental
    ... available evidence and you being in possession of none?) ... person would come to with the evidence that is known. ... What I said is that the evidence is stacking up for the IPCC not being any better than the PCA. ... I *still* think it was crazy for them not to interview him. ...
    (uk.legal)
  • Re: IPCC rules Forest Gate shooting accidental
    ... better than the PCA. ... available evidence and you being in possession of none?) ... person would come to with the evidence that is known. ... not been made public," is extremely dishonest. ...
    (uk.legal)