Re: optimization of sum of weighted bernoulli params

From: Ray Koopman (koopman_at_sfu.ca)
Date: 08/07/04

  • Next message: Richard Ulrich: "Re: Help- Statistical terms/concepts in layman definitions" 
    Date: 6 Aug 2004 19:17:06 -0700

    Hal Daume III <hdaume@isi.edu> wrote in message
    news:<Pine.LNX.4.44.0408052120390.27666-100000@albini.isi.edu>...
    > I have an optimization problem that I'm having difficulty solving.
    > I'd greatly appreciate any thoughts people have on the issue.
    > The expression in question is of the form:
    >
    > Q(a) = sum_n w_n prod_i (a_i)^(x_ni) (1-a_i)^(1-x_ni)
    >
    > where w_n is a positive real value, and each x_ni is either 0 or 1.
    > The product is clearly the probability of the xs under a bernoulli
    > distribution with paramter vector a. i need to find a vector a that
    > maximizes Q. the problem, of course, is that because of the stupid
    > sum, taking logs doesn't separate the a_is.

    Have you considered minimizing -log Q with a garden-variety brute-force
    multiparameter minimizer? (Yes, I know the derivatives will be messy).

    For initial estimates, try a_i = (sum_n w_n x_ni)/(sum_n w_n).

    If any such initial a_i is 0 or 1 then it and all its x_ni can be
    deleted and the problem size reduced.

    To maintain 0 < a < 1, write t = log(a/(1-a)) <==> a = 1/(1+exp(-t)),
    and minimize with respect to t.

    [posted & e-mailed]

  • Next message: Richard Ulrich: "Re: Help- Statistical terms/concepts in layman definitions"