Re: using data for Bayesian prior

From: JRKRideau (JohnKane9996_at_hotmail.com)
Date: 11/18/04 

Date: 18 Nov 2004 07:07:31 -0800

hrubin@odds.stat.purdue.edu (Herman Rubin) wrote in message news:<cmvuo0$5lpe@odds.stat.purdue.edu>...
> In article <cmvgag$6ec$1@planja.arnes.si>,
> Aleks Jakulin <a_jakulin@@hotmail.com> wrote:
> >Geert Verdoolaege wrote:
> >> I am not sure why exactly, in Bayesian analysis, it is a bad idea to
> >> use information on the measured data for the construction of a prior
> >> distribution of the parameters.
>
> For a mechanical Bayesian, this is correct. If you have an
> infinitely fast computer with zero cost, you could take any
> combination of prior, likelihood, and cost and compute the
> Bayes procedure.
>
> >The basic explanation is that you're double-counting the evidence.
> >First, you use the data to construct a prior, and then you update the
> >prior with the *same* data.
>
> There is another way of looking at it which points out that
> this is not exactly what is being done. It could be that
> the true prior is essentially incomputable, but if one
> looks at it as a prior on computable priors, one can
> estimate the computable prior from the data, and use this
> to compute the action to be taken. In the case of presumably
> independent estimation, there are even Bayes empirical Bayes
> procedures.
>
> Otherwise, this is not optimal, but it may be close, and
> it is possible to get general results here.
>
> >The empirical Bayes procedure uses a part of the data to construct the
> >prior, and the other part of the data to update it. This is
> >reminiscent of cross-validation, where you use a part of the data to
> >construct a model, and the other part to examine how well the model
> >fits the data that was not used for the model.
>
> See the above. I do not agree with the above; the entire
> data can be used for both, and usually should be.
>
> >However, hard-core Bayesians do not like empirical Bayes very much
> >because they see it as an approximation to hierarchical Bayesian
> >analysis. There, you don't specify the prior, but instead specify a
> >model of the prior's parameters.
>
> What is important is the risk of the procedure. There are
> those who use computationally simple, but very definitely
> unreasonable, priors, often based on the data, and look at
> the problem of approximating the prior. This may or may
> not be a reasonable way to look at it. When testing a null
> hypothesis with a reasonable amount of data, the prior
> probability that the hypothesis is true turns out not to be
> of importance at all. Looking at the Bayes risk shows this
> to be the case.
>
> Many of the "hard-core" Bayesians seem to have no qualms in
> choosing a convenient prior which cannot be reasonable, and
> often depends on the form of the experiment. This violates
> the consistency assumptions, which call for minimizing the
> prior Bayes risk, treating the "prior" as weights, and not
> as probabilities. The loss and prior cannot be operationally
> separated, as only the product enters. Using the data to
> estimate both may well be needed for high-dimensional problems;
> this includes almost all of the so-called "nonparametric" ones.

I have never used Bayesian stats and am certainly not a statistican.
Can anyone recommend a "Bayesian Stats for Dummies" text to give me a
feel for them?

John Kane
Kingston ON

Relevant Pages

  • Bayesian continued and shuffling
    ... valid prior history predictable by Bayesian theorem. ... For the record I try not to adjust and keep using pure probability ... Every so often I see a reference to Bayes' ...
    (rec.gambling.poker)
  • Re: The danger of classical hypothesis and significance tests [was Re: MADLY AMUSED]
    ... Given the pained and heartfelt way in which Jaynes likened himself to Galileo going up against the classical orthodoxy, I would have thought a Bayesian such as you would refrain from taking up the sort of position you here do, namely that of self-appointed Defender of the Faith, taking aim at what you too quickly deem to be misguided crackpots. ... The problem, which you pretend not to see or understand, is that which has beset the Bayesian approach from the very beginning -- namely the justification, or lack, for the notion of prior. ... the only question at issue is what calculus should be applied to the likelihood function for purposes of marginalization and of change of variable. ... The essential insight derives from fuzzy set theory, which is to recognize that the likelihood function minimally satisfies the conditions of the membership function of a fuzzy set, and more fundamentally that uncertainty in a model parameter is of a different sort than uncertainty in the next occurrence of a random variable. ...
    (sci.stat.math)
  • Re: Highest Posterior Density
    ... You need to be able to supply the Bayesian ingredients of a prior ... obtain your posterior distribution of your PARAMETER ... you do not need to perform the integration as the ...
    (sci.stat.math)
  • Re: Bayesian continued and shuffling
    ... From a Bayesian point of view did I not use prior knowledge? ... An observation about hand shuffling: ... For the record I try not to adjust and keep using pure probability ...
    (rec.gambling.poker)
  • Re: Where it all began: Illywhackers lack of understanding about Bayesian inference
    ... RF> maximum of the posterior distribution, ... RF> bother with Bayesian inference? ... RF> Never even heard of the acronym MAP in Bayesisian inference. ... It is called a "diffuse prior" not a uniform prior. ...
    (sci.stat.math)