EM applied to gaussian mixture model
- From: kingpin@xxxxxxxxxxx
- Date: 24 Apr 2007 14:13:21 -0700
Hi,
i'm implementing the EM algorithm for gaussian mixtures (in R^2) but
i'm having serious problem with it.
I will do an example.
In R^2 we have three data points
x1=(1,2),x2=(1,1),x3=(-2,1)
I use 2 components, with mean m1 = x1=(1,2), m2=x3=(-2,1) with
weights w1=1/2, w2=1/2
(generally speaking, every component's mean will be set to one of the
dataPoints, chosen in a random way).
Now i must initialize the covariance matrixces, but i don't know how
to do it.
I have read that i can "select the covariance matrix of the whole data
set for each
of the initial K covariance matrices",where K is the number of
components, but i don't know how the covariance matrix for the whole
data set is defined.
Finally, in the M-step, there is the update of the covariance
matrices. I'm referring to
http://www.stat.wisc.edu/~mchung/teaching/stat992/lec33.pdf
In that formula, the author write
(x(i) - m_k)(x(i) - m_k)^t
where m_k is the updated mean of the component k, x(i) is the i-th
data component (this is only a part of the formula). I think that this
vector operation is the Kronecker vector product (see
http://en.wikipedia.org/wiki/Tensor_product).
Is this right?!
What if the determinant of one of the covariance matrices becomes 0? I
know that they are
positive definite (so their determinant should be always > 0), but i
have this problem.
There are two reason:
1.maybe i setup the covariance matrices in the wrong way
2.maybe i update the covariance matrices in the wrong way
I think that all the other step are implemented in the right way.
Help please!!!
Source code, web sites, books, suggestions are all welcomed!!
Thank you.
.
- Prev by Date: Re: future of statistics
- Next by Date: Re: Simple stat question that shouldn't be posted here
- Previous by thread: time changing distributions, what is significant ?
- Next by thread: MLE and stochastic volatility modelling
- Index(es):
Relevant Pages
|
|
