Re: different priors (flat, uniform, etc)


DZ wrote:
Anon. <bob.ohara@xxxxxxxxxxxxxxxxx> wrote:
Reef Fish wrote:
David Winsemius wrote:
"Reef Fish" wrote
Can we hear a bit more about how is Beta(1,1) is an informative prior for a
binomial problem?

It CHANGES the likelihood function to form the posterior distr.

My one-line response turned out to be more succinct and penetrating
than I had thought, because they is the KEY to any PROPER prior
that is informative!
[...]

Intriguingly, Reef Fish also made this comment on this thread:
RF> The posterior distribution is the likelihood function if the prior
RF> is "diffuse" (which is NOT the same as a "uniform" or "flat" prior).

So, apparently the beta(1,1), which is also the uniform distribution, is
"diffuse" but not "uniform".

No, the uniform distribution is hardly diffuse. It is uniform AND
informative,
as I had said before.

I was out of town on the weekend. I sort of took advantage of that to
see what Bayesians I can flush out of the wood work, to show, without
any doubt, that John Uebersax and Anon Bob O'Hara were definitely
NON-Bayesians and that they were completely wrong, as I had indicated
with the few hints I gave.

The first one to surface was Herman Rubin, who mentioned some points
others followed up on, but Herman misunderstood the statement I made
about "conjugate priors" (which I corrected this morning).


David Winsemius indicated he made SOME efforts to READ what's
relevant. When he showed that he read the Edward, Lindman, and
Savage paper, I was TEMPTED to explain to him what the score
was, since he wasn't being confrontational even though his original
post right after Anon Bob (even after my explanation) seemed to
indicate that he never read a Freshman's BOOK about Bayesian
inference, and he STILL hasn't, or else he would have solved the
mystery himself. So, I'll reveal the Da Vinci Code to him and
all when I get to his post in the afternoon, following up on Herman
Rubin's comments on his questions which I didn't answer.

Then DZ emerged. I think that pretty much exhausted ALL the
educated Bayesians in sci.stat.math. from what I can gather in
my reading this group for 1 1/2 years.

DeGroot's comment on Shafer's paper "Lindley's paradox" criticized the
idea that "diffuse" should mean equal probability for all parameter
values and that in the normal(m,s) case, "diffuse" implies, more
appropriatly, that for example m^2 might be large - that is, the
variance is large.

That is one of the meanings of the term "diffuse", and the normal
example (with a normal likelihood) is a GOOD example to say that
you CANNOT have a uniform distribution over the entire real line!
But it says more than that. It's related to Savage's "principle of
stable estimation" which gave a very quantifiable meaning to the
meaning of diffuse in the sense of "locally uniform" over a
likelihood function that is very sharp.

I had used the slightly altered and simplify meaning of "diffuse"
prior to mean one that would leave the posterior exactly the
same as a normalized likelihood function, so that the non-Bayesian
MLE becomes the maximum point of the posterior for a Bayesian,
if the likelihood function and the unnormailized posterior coincide.

Similarly, in the beta prior case, beta(1,1) is
uniform, but may not be diffuse enough, because as you let both a,b
starting from beta(a=1,b=1) go to zero, the variance increases.

That is one way to look at it. But THIS was what I pointing at, for
the Freshman textbook nobody seemed to have found for the Da
Vinci Code of the conjugate prior beta for the binomial p.

The CONJUGATE part means both the prior and the posterior
are members of the beta family. If the prior distribution of the
binomial p is beta( alpha, beta ), and r and (n-r) are the
powers of p and (1-p) in the likelihood function, then the
posterior parameters will be changed to (alpha + r) and
(beta + n - r), in the beta family!

This needs one more step of explanation to show why Anon
Bob O'Hara was looking at BOTH the beta(1,1) prior and the
likelihood function and STILL missed it! That was the proof
that Bob O'Hara had never seen that Freshman book either
or any book, on how to make a Bayesian inference of the
parameter p of a Bernoulli process or a Binomial distribution.

I hope SOME ONE can manage to find a Bayesian book
(the more elementary the better) and show us what happens
when a uniform prior Beta(1,1) is applied to the Binomial
problem of p given r successes and f failures, r + f = n.

Meanwhile, I'll take a short break before explaining it in my
reply to David Wisenmius's latest post of Sun, Oct 29 2006
1:53 pm, which contain both Herman Rubin's reply
yesterday, and a very relevant webpage provided by David.

Stay tuned.

-- Reef Fish Bob.

.

Relevant Pages

  • Re: The danger of classical hypothesis and significance tests [was Re: MADLY AMUSED]
    ... Given the pained and heartfelt way in which Jaynes likened himself to Galileo going up against the classical orthodoxy, I would have thought a Bayesian such as you would refrain from taking up the sort of position you here do, namely that of self-appointed Defender of the Faith, taking aim at what you too quickly deem to be misguided crackpots. ... The problem, which you pretend not to see or understand, is that which has beset the Bayesian approach from the very beginning -- namely the justification, or lack, for the notion of prior. ... the only question at issue is what calculus should be applied to the likelihood function for purposes of marginalization and of change of variable. ... The essential insight derives from fuzzy set theory, which is to recognize that the likelihood function minimally satisfies the conditions of the membership function of a fuzzy set, and more fundamentally that uncertainty in a model parameter is of a different sort than uncertainty in the next occurrence of a random variable. ...
    (sci.stat.math)
  • Re: different priors (flat, uniform, etc)
    ... RF> The posterior distribution is the likelihood function if the prior ... Vinci Code of the conjugate prior beta for the binomial p. ...
    (sci.stat.math)
  • Re: Anyone found an Elementary Bayesian Book yet?
    ... prior, and I regret doing so now, because it introduced the ... RF> those in the likelihood function. ... RF> because the posterior is given by ...
    (sci.stat.math)
  • Re: different priors (flat, uniform, etc)
    ... Now that we have the Bayesian recipe resolved about the problem ... the beta distribution is a member of the ... RF> conjugate prior family -- meaning both the prior AND posterior ... likelihood function as the posterior distribution. ...
    (sci.stat.math)
  • Re: Help needed with expletives
    ... RL> It is, in fact, precisely what was being worked toward prior ... RL> to the blowup. ... The uniform, concerted, and poisonously ... RL> vociferous response of The "Gay Community" has been and ...
    (rec.arts.sf.composition)