Re: Beyond simple penalized regression

Jerry Dallal wrote:
Herman Rubin wrote:
In article <nPudnbGNB4ev7xfYnZ2dnUVZ_sC3nZ2d@xxxxxxxxxxx>,
Jerry Dallal <gdallal@xxxxxxxxxxxxxxxxxxxx> wrote:
Bob O'Hara wrote:
Jerry Dallal wrote:
JS wrote:
On Dec 20, 1:41 pm, Jerry Dallal <gdal...@xxxxxxxxxxxxxxxxxxxx> wrote:
I'll *give* you computational convenience. I've always considered that
a distraction. When the Gibbs sampler and MCMC hit the collective
consciousness many thought Bayesians' problems solved since it now
became feasible to use something more flexible than a conjugate prior.
However, computational feasibility is NOT where the problem is. It's this:

Either the prior matters or it doesn't. Well, it does.

But is assuming a prior any more dangerous than assuming linearity, or
normal residuals, or some other common artificiality.

Yes, and, unfortunately, the question is damning. Linearity, normal residuals, and the like can be checked. "Checking a prior" is an oxymoron. The question is damning because a prior is not something that is "assumed". It *is*.

No, it is a model assumption. Like any model assumption, one can still check it. One could simply choose another prior, and see if the the results are different if the alternative prior is used.

Bob

I'll reply to both Bob and JS here. I'm not sure what kind of Bayesianism you're practicing. I understand how to check model assumptions, but if checking one's prior is not an oxymoron, then the type of Bayesianism you're suggesting becomes nothing more than a self-fulfilling prophecy.

If the rule is that a prior is okay if the results don't change when the prior changes, then it's not a real prior. The whole point of Bayes methods is that the prior DOES matter. Think about it. If the chief complaint about frequentist methods is that they don't incorporate prior opinion, then what kind of an alternative is it that incorporates prior opinion only if a person's particular prior doesn't change anything?

What you fail to realize is that many aspects of a prior do not
affect the decision, or not much.

Oh, I recognize it, but I also recognize that there are many aspects of a multidimensional prior that DO matter. Even if it were true that most aspects didn't matter, it's the ones that do that are more important. A single weak link is all it takes for things to fall apart.

If priors are always robust or the evidence is always overwhelming, then Bayes methods are bogus. Bayes methods are only of value if they can be used to update a nontrivial prior to produce a nontrivial posterior. I've yet to be convinced that it is possible to assess such priors.

I think nowadays most people who use Bayesian methods do so because they make it much easier to build and fit the complex models we are interested in, so priors are less of a concern. Because it's all just probability theory, it's easy to do things like integrate out parameters we are not interested in. Some of the things I've done were reasonably straightforward in the Bayesian framework, but I've no idea how I would have handled them using a Frequentist setup.

Go on, Jerry, try it. You might like it. :-)

Bob
.

Relevant Pages

  • Re: Beyond simple penalized regression
    ... "Checking a prior" is an oxymoron. ... I understand how to check model assumptions, but if checking one's prior is not an oxymoron, then the type of Bayesianism you're suggesting becomes nothing more than a self-fulfilling prophecy. ... Oh, I recognize it, but I also recognize that there are many aspects of a multidimensional prior that DO matter. ... If priors are always robust or the evidence is always overwhelming, then Bayes methods are bogus. ...
    (sci.stat.math)
  • Re: Beyond simple penalized regression
    ... computational feasibility is NOT where the problem is. ... Either the prior matters or it doesn't. ... I understand how to check model assumptions, but if checking one's prior is not an oxymoron, then the type of Bayesianism you're suggesting becomes nothing more than a self-fulfilling prophecy. ... If the chief complaint about frequentist methods is that they don't incorporate prior opinion, then what kind of an alternative is it that incorporates prior opinion only if a person's particular prior doesn't change anything? ...
    (sci.stat.math)
  • Re: Beyond simple penalized regression
    ... computational feasibility is NOT where the problem is. ... Either the prior matters or it doesn't. ... I'll reply to both Bob and JS here. ... Bayesianism you're practicing. ...
    (sci.stat.math)
  • Re: Beyond simple penalized regression
    ... became feasible to use something more flexible than a conjugate prior. ... Bayesianism you're practicing. ... are those of the Statistics Department or of Purdue University. ... Herman Rubin, Department of Statistics, Purdue University ...
    (sci.stat.math)
  • Re: Beyond simple penalized regression
    ... "Checking a prior" is an oxymoron. ... I understand how to check model assumptions, but if checking one's prior is not an oxymoron, then the type of Bayesianism you're suggesting becomes nothing more than a self-fulfilling prophecy. ... Oh, I recognize it, but I also recognize that there are many aspects of a multidimensional prior that DO matter. ... If priors are always robust or the evidence is always overwhelming, then Bayes methods are bogus. ...
    (sci.stat.math)