Re: Relative Entropy

From: illywhacker (illywhacker_at_free.fr)
Date: 02/26/05 

Date: 26 Feb 2005 05:05:05 -0800

This certainly depends on how you take the limit, i.e. the relationship
between the values of p and q as you take the limit.

However, again at least for finite sets, there is no difference in
practice between a distribution p on some set X and one on the set Z =
X \cup Y if p(Y) = 0. Of course, if q is defined on Z, and q(Y) \neq 0,
you need to include Y. In general, you can restrict p and q to the set
of elements on which one of them in non-zero.

I think the general answer is that you can effectively exclude elements
of the sigma algebra that have measure zero.

illywhacker;