Re: Howto Initialize Parameter of Gamma Mixture Densities for EM

On Jul 7, 4:36 am, gundalav <gunda...@xxxxxxxxx> wrote:
Hi all,

Is there any reasonable way to initialize the gamma parameters for
maximum
likelihood estimation using EM.

Any advice, pointer to paper URLs are greatly appreciated.

Some people mention this can be done
by "drawing mixing proportions from Dirichlet distribution".

But that is vague for me on how it is done and why.
Please advice.

Drawing from a Dirichlet would seem to be related to drawing some
initial values for the mixture weights of your model. You also should
consider ways of initializing the shape and scale parameters of each
gamma component. Some ideas of the top of my head assuming your
mixture model is written as

K
p( x ) = SUM w_k Gamma( alpha_k, beta_k ) where SUM w_k = 1
k=1

For the mixture weights

1. Sample from a Dirichlet distribution with different alpha values
that represent your belief about the relative strength of the
components. Set all the alphas equal if you believe each
component
to have approximately equal weight. Make alpha very large to
reduce the variance, i.e. Dirichlet([1000,1000,1000]) will
produce
a vector very close to [1/3 1/3 1/3].
2. Set all the mixture weights equal, i.e. for K components w_k = 1/
K

For the gamma parameters

1. Pick random alpha and beta values from some range (probably bad)
2. Fit a single Gamma distribution to the data and jitter the
parameters
3. Fit a Gamma distribution to a random subset of the data. Repeat
K times.\
4. Run a kernel density smoother. Pick the K highest modes. Set
the mode of
the kth Gamma distribution equal to the kth empirical mode. The
variance
is a free parameter.
5. Place a prior over the Gamma parameters and sample K times (like
(1) but
a bit more formal)


Since you're doing EM I assume you've introduced unknown assignment
variables z_ik that indicate if observation i was generated by
component k and that the E-step computes E[z_ik] for all i and k. If
this is the case, you could also randomly initialize E[z_ik] s.t.
SUM_k E[z_ik] = 1 and then begin the EM iterations with the M-step.

-Lucas
.

Relevant Pages