Re: how to compare the Gaussian probability of data with different dimensions?

On Mar 22, 5:07 am, Richard Ulrich <Rich.Ulr...@xxxxxxxxxxx> wrote:
On 21 Mar 2007 02:45:29 -0700, "illywhacker" <illywac...@xxxxxxxxx>
wrote:
On Mar 21, 3:31 am, Richard Ulrich <Rich.Ulr...@xxxxxxxxxxx> wrote:
On 19 Mar 2007 12:05:45 -0700, "zl2k" <kdsfin...@xxxxxxxxx> wrote:

hi, all
Suppose I have a data set with data having m-dimensions, and I have a
multivariate Gaussian (m dimensional) to describe it. In some cases,
the covariance matrix could be singula and I have to deduce the
dimension by projecting the data to a lower dimension to calculate
the probability. My question is: how can I compare the probability
using m dimension with using (m-k) dimension? (usually k=1) The change
of the dimensionality is only because of the singula covariance
matrix.
What I am thinking is that the data using lower dimension will have
higher probability than using higher dimension. How can I make
adjustment to compensate of change of the dimensionality? Thanks for
help.

Illy has given a long response, which seems okay.

Here is a short response. If you are comparing
likelihood functions - which is what it sounds like
and looks like from the later example - you might
look up AIC and BIC.

--
Rich Ulrich, wpi...@xxxxxxxxxxxx://www.pitt.edu/~wpilib/index.html

This would be good for model estimation, but if I understand
correctly, the OP assumes that the models are already known. Now the
point is to classify new data.

If you figured out what he is doing, you are ahead of me.

Conventionally, if he were talking about "probability" for
classifying, he would use the tail probabilities -- but I do not
see how that fits his description. He is mis-describing
likelihoods as probabilities, so far as I could tell.

He should calculate the probablity of the different classes given the
data, which is what I was saying in my post. (I hope you are not going
to tell me that the 'probability of a class' makes no sense because
'class' is not a 'random variable' but simply an unknown
quantity :) !)

He may be misusing the word 'likelihood' slightly, but it is more
useful as a term for the probability distribution of the data (which
may also be a function of other parameters) than as a term for this
quantity when viewed solely as a function of those parameters, a
distinction that has no mathematical use as far as I can see.

I don't know what he gets if he normalizes each term by
dividing into the Maximum likelihood. Is there a correction
for number-of-parameters, like when using AIC or BIC?

There is no need for ad hoc solutions: probability theory tells you
what to do. If he were learning the model covariance, then he might
want to add extra weight to the prior proobability of a singular model
if that were reasonable given his context, but once the models are
learned and one of them is singular, then as far as I can see the game
is over. If the data point to be classified is on the hyperplane, it
comes from the class with the singular distribution; if it is not on
the hypersurface it comes from the class with the non-singular
distribution. This is true as the limit of non-singular distributions,
and so makes sense.

illywhacker;

.

Relevant Pages

  • Re: Spectrum, Markov, Large Deviations, and PI
    ... The connection of Probability-Statistics with volume and Lebesgue ... elementary or even first upper division probability or statistics ... Volume, area, and length in Euclidean ... capacity dimension on closed bounded (that is ...
    (sci.stat.math)
  • Re: Sci-Fi Noein - "Shrodingers Sand Box"
    ... the physics sound more complicated than it really was. ... we would need a complicated set of probability functions for our particles. ... dimension they want to reach. ...
    (rec.arts.anime.misc)
  • Re: Black Hole Bouncing vs Evaporation vs Control
    ... statistics, randomized decision rules are key in statistical decision ... words about Probability as a dimension after writing the new equation: ... It is typical to regard Probability as dimensionless because of the ...
    (sci.physics)
  • Re: Dimensional Analysis Via PI Versus Factor/Cluster/Scaling/PCA
    ... >danger and decision to move toward ... Mathematical sociology and mathematical psychology should now ... for not assigning a dimension to probability is the "relative freq- ... a number divided by a number which has no dimension. ...
    (sci.stat.math)
  • Re: Arbitrarily Many Nested Loops
    ... Sinan Unur ... Neither the code snippet above nor your verbal description makes any ... in another vector (one size per dimension). ... distribution where the success probability differs between trials. ...
    (comp.lang.perl.misc)