Re: Explanation of Maximum Entropy

From: Radford Neal (radford_at_cs.toronto.edu)
Date: 08/29/04 

Date: 29 Aug 2004 15:13:35 GMT

>> If it helps, we can focus the question by posing the specific
>> of coinflipping. If one observes 6 heads out of 10 flips of a
>> potentially biased coin, then one can use standard Bayesian methods...
>>
>> How would this be affected according to a Maximum Entropy approach?

In article <cgl2ei$mso$1@planja.arnes.si>,
Aleks Jakulin <a_jakulin@@hotmail.com> wrote:
>
>With the constraint of having two outcomes, MaxEnt would yield you the
>PMF of [0.5,0.5].

Why? The data is 6 heads out of 10, not 5 out of 10. Shouldn't the
best distribution pay at least a little attention to the data?

I suspect you have in mind some sort of frequentist test that fails to
reject the null hypothesis that the distribution is [0.5,0.5], and
since this distribution has maximum entropy, you go for it. Of
course, you WILL reject this null if you set the significance level
high enough. So the maximum entropy result, often touted as being an
"objective" solution to the problem, actually depends on a totally
arbitrary choice of significance level.

   Radford Neal

Relevant Pages

  • Re: Explanation of Maximum Entropy
    ... John Uebersax wrote: ... > If the prior distribution is uniform, ... > How would this be affected according to a Maximum Entropy approach? ... define the class intervals such that each has kl/j observations; ...
    (sci.stat.math)
  • Re: skewed normal distribution
    ... (see also Maximum entropy probability distribution in wikipedia) ... So the maxent distributions give best fit pdfs for any number of moments. ...
    (sci.stat.math)
  • Re: Explanation of Maximum Entropy
    ... They standard maximum entropy approach is to find the distribution ... that the expectations have these values. ... Dept. of Statistics and Dept. of Computer Science radford@utstat.utoronto.ca ...
    (sci.stat.math)
  • Explanation of Maximum Entropy
    ... I understand that maxent may have advantages in certain applications, ... applicable innovation--or revolution--within statistics). ... If the prior distribution is uniform, ... How would this be affected according to a Maximum Entropy approach? ...
    (sci.stat.math)
  • Re: Dynamical Systems and Expansion-Contraction
    ... maximum entropy corresponds to the ... distribution is the "first cousin" of the uniform distribution, ... normal/Gaussian distributions for the continuous case aren't "ranked" ...
    (sci.physics)