Re: Discriminant analysis - Normality assumption

From: Aleks Jakulin (a_jakulin_at_@hotmail.com)
Date: 07/29/04 

Date: Thu, 29 Jul 2004 21:53:32 +0200

Jay Liu wrote:
> In fact I've seen many people saying in their papers that Fisher's
> DA or PCA are based on the normality assumption. This is absolutely
> wrong. They should read original papers.

It is not wrong. One has to be specific, though. This claim refers to
optimality. Normality assumption implies that the PCA is an "optimal"
reduction of dimensionality as measured through variance or
likelihood. If you don't assume normality, PCA is not necessarily an
optimal reduction of dimensionality. Let me give you an example. If
you have a mixture of Gaussians as your data, PCA is not going to be
an "optimal" dimension reduction method. Instead, the mixture of
Gaussians itself is going to be the optimal dimension reduction
method.

Claims that PCA is optimal prevent people from a) thinking about what
model to use for their data and b) thinking probabilistically. I
consider this harmful.

Other than optimality, both significance testing (as others mentioned)
and also parameter variance estimates are dependent upon this
assumption.

Aleks 

-- 
mag. Aleks Jakulin
http://www.ailab.si/aleks/
Artificial Intelligence Laboratory,
Faculty of Computer and Information Science, University of Ljubljana.

Relevant Pages

  • Re: Discriminant analysis - Normality assumption
    ... Normality assumption implies that the PCA is an "optimal" ... Gaussians itself is going to be the optimal dimension reduction ... Other than optimality, both significance testing ...
    (sci.stat.math)
  • Re: Probability of weights in a PCA space?
    ... I've seen it mentioned in a couple of papers that the PCA of a dataset ... You can't convert a non-normal distribution of multiple ... Using the assumption of normality, the Mahalanobis distance ...
    (sci.stat.math)