Re: same entropy between two probability distribution functions.

From: Yan ZHANG (buaanupt_at_sina.com)
Date: 01/21/05 

Date: Fri, 21 Jan 2005 11:07:25 +0800

Thank you very much for your comments.

Could you plz give more explanation why using the same entropy as the
criteria is not a good idea? 

-- 
Best Regards
Yan ZHANG
http://www.nict.com.sg/zhang/
"Hiu Chung Law" <antispam@antispam.org> wrote in message 
news:csosnq$qcq$2@msunews.cl.msu.edu...
> In sci.stat.math Yan ZHANG <buaanupt@sina.com> wrote:
>> Suppose that a non-negative random variable X follows probability density
>> function f_X(t). Since f_X(t) is too complicated, I would like to
>> approximate f_X(t) by simpler function f_X2(t) such as exponential or
>> hyper-exponential distribution.
>
>
>> I was once suggested that f_X(t) can be approximated by f_X2(t) with the
>> same entropy. Can you please give some references, weblink or source code
>> discussing this technique? Thank you very much.
>
>
>> -- 
>
>> Yan ZHANG
>> http://www.nict.com.sg/zhang/
>
>
> I would be surprised if approximation by entropy is a good idea.
> Anyway, there is too little information to make a useful suggestion,
> because the forms of f_X(t) and f_X2(t) are unknown.
>
> You may want to check out variational method which finds the
> "best" f_X2(t) within a given family to approximate f_X(t)
> with respect to KL divergence.
>
> P.S. Less relevant newsgroups have been removed from "Followup-To".