EM applied to gaussian mixture model

Hi,

i'm implementing the EM algorithm for gaussian mixtures (in R^2) but
i'm having serious problem with it.

I will do an example.

In R^2 we have three data points

x1=(1,2),x2=(1,1),x3=(-2,1)

I use 2 components, with mean m1 = x1=(1,2), m2=x3=(-2,1) with
weights w1=1/2, w2=1/2
(generally speaking, every component's mean will be set to one of the
dataPoints, chosen in a random way).

Now i must initialize the covariance matrixces, but i don't know how
to do it.

I have read that i can "select the covariance matrix of the whole data
set for each
of the initial K covariance matrices",where K is the number of
components, but i don't know how the covariance matrix for the whole
data set is defined.

Finally, in the M-step, there is the update of the covariance
matrices. I'm referring to

http://www.stat.wisc.edu/~mchung/teaching/stat992/lec33.pdf

In that formula, the author write

(x(i) - m_k)(x(i) - m_k)^t

where m_k is the updated mean of the component k, x(i) is the i-th
data component (this is only a part of the formula). I think that this
vector operation is the Kronecker vector product (see
http://en.wikipedia.org/wiki/Tensor_product).

Is this right?!


What if the determinant of one of the covariance matrices becomes 0? I
know that they are
positive definite (so their determinant should be always > 0), but i
have this problem.
There are two reason:

1.maybe i setup the covariance matrices in the wrong way
2.maybe i update the covariance matrices in the wrong way

I think that all the other step are implemented in the right way.

Help please!!!

Source code, web sites, books, suggestions are all welcomed!!
Thank you.

.

Relevant Pages