Re: Anyone found an Elementary Bayesian Book yet?

Reef Fish wrote:
David Jones wrote:
Reef Fish wrote:
Thanks. I do not have any books nor access to any book.

But in my reply to Russell Martin who cited the book,
_Statistical Inference and Prediction in Climatology: A
Bayesian Approach_ by Edward Epstein

http://groups.google.com/group/sci.stat.math/msg/1312f5f2dee4756d

you should find something very similar (if not identical) in your
book on the relation between the likelihood, prior, and posterior
in the case of the conjugate Beta prior on the problem of
Bernoulli
p:

"no brainer" of using the Beta as the conjugate prior,
because the posterior is given by (double primes)

r" = r + r' and n" = n + n'.


It gives, after re-expression:

likelihood : Beta(r+1, n-r+1)

I should have included the explanation here. The likelihood
has only r and n-r in the exponent.

But the L(p|r) is NOT in form of a Beta density!
The kernel of the Beta density has (alpha -1) and
(beta - 1) in the exponents!

That's why, in the form of a Beta density, the r and n of
the likelihood function must be parametrized as (r+1)
and (n-r+1). The posterior BETA, from the Beta Prior
(alpha, beta) will be Beta (r+1+ alpha, n-r+1+beta),
which is why alpha and beta need to be both ZERO
for the posterior to be

Beta(r+1, n-r+1) which is the original likelihood function
p^r (1 - p)^(n-r)

(I had to correct a couple of places above where "-r" was
left out. But the above as it stand now, is the correct version.

prior : Beta(alpha,beta)

posterior : Beta(alpha+r, beta+n-r)

Sorry, this should have been Beta(r+1+alpha, n-r+1+beta)

Well, no. Consider only the powers of p. The likelihood function that
comes originally from the Binomial(r,n), equivalently Beta(r+1,n-r+1),
contributes has p to the power r. The prior distribution that is
Beta(alpha,beta), has p to the power (alpha-1). So the posterior has p
to the power (r+alpha-1), which is the term that would arise from a
Beta(alpha+r, beta+n-r) distribution.

Another check is to see what happens when there is no data from the
experiment, so that the posterior is the same as the prior. "No
experiment" would be equivalent to n=r=0



I also should have used the notation I had used elsewhere in
using ro as the r in the Beta density and r as the same ro in the
likelihood function. (Similarly for no as n+1 in the Beta, and
n in the likelihood). That would have eliminated not only
the ambiguity but the propensity for typos.

Sorry about that, but I am VERY GLAD that you found the
typo errors to make the same error Bob O'Hara did, so that
I can correct you. :-) At the same time, you would NOT
have made the error (and probably shouldn't even with my
typo) had I not made the typo error in the Beta index,
because the ESSENSE of the "cook book recipe" is that
you add the indices of the BETA in the likilihood and the
BETA in the prior to get the BETA in the posterior.


Actually, the books I see don't use the Beta distribution for the
likelihood, rather the Binomial, and you amy be thinking of adding the
indices of the Binomial and Beta in some sense. For the Beta's, you
need to add the indices and subtract one.

David Jones


.

Relevant Pages

  • Re: different priors (flat, uniform, etc)
    ... RF> The posterior distribution is the likelihood function if the prior ... Vinci Code of the conjugate prior beta for the binomial p. ...
    (sci.stat.math)
  • Re: different priors (flat, uniform, etc)
    ... RF> The posterior distribution is the likelihood function if the prior ... Vinci Code of the conjugate prior beta for the binomial p. ...
    (sci.stat.math)
  • Re: Anyone found an Elementary Bayesian Book yet?
    ... book on the relation between the likelihood, prior, and posterior ... in the case of the conjugate Beta prior on the problem of Bernoulli ...
    (sci.stat.math)
  • Re: probit analysis with natural response
    ... Or ensure that in these cases the likelihood is large/ ... fr to give the negative log-likelihood. ... -Inf, consequently, mle() will give a error message, if it was not ... So use a positive starting value for beta! ...
    (sci.stat.math)
  • Re: Help on fminsearch
    ... on the coefficient of the covariates. ... coefficients, ie. beta = onesand won't do the optimisation. ... INSIDE the myfun function? ... I don't want to load the data in the function, but the likelihood is ...
    (comp.soft-sys.matlab)