Re: Beyond simple penalized regression


JS wrote:
On Dec 18, 4:03 pm, "Reef Fish" <large_nassua_grou...@xxxxxxxxx> wrote:
Ridge Regression is certainly nonBayesian, and has been out of favor since the 1980s
for the inappropriate reasons (wrong sign of regression coefficient
by those who don't know what regression signs mean) for which the
procedure was used.

I don't know what to make out of this, except that you are unfamiliar
with the equivalence of ridge regression and exchangeable gaussian
prior on the regression coefficients.

Your inference is correct. Both of these are useless to me. Ridge
Regression can also be given some pseudo-Bayesian interpretations,
but that doesn't make it any more useful or legitimate as a statistical
method.


OP> I'm mainly after an automatic way to choose the
OP> accuracy of the prior.

That is completely ANTI-Bayesian.For the simple reason that a "prior" is supposed
to represent the user's OPINION about the parameters, not something automatically
chosen from DATA.

I could also have said "I would like to express my vague prior
knowledge I'm willing to put into this as a hyperprior for the gaussian
rather than as a fixed gaussian prior", if it makes you feel better.

No. It make you and other nonBayesians feel like they should demand
respectibility by throwing the term Bayesian around because it seems
fashionable in some circles. You seemed to be completely unaccustomed
to the fact that priors are NOT to be taken likely by a true Bayesian.
All your terms are just convenient concepts for the lazy-non-thinking
non-Bayesians who think they are using Bayesian methods.


This is a context where the prior information wanted on the model is on
a very general level, like "I find large coefficients on collinear
predictors unlikely".

Why pretend to be a Bayesian? Why not just use legitimate nonBayesian
regression methods?

The fact that YOU can make the statement

JS> "I find large coefficients on collinear predictors unlikely".

NOT knowing what each and everyone of all the predictors used in the
regression is a 100% unmistakable indication that you DON'T KNOW
how to interpret the coefficients in a regression. The fact that you
think "collinear predictors" unlikely when you have even dozens,
let along hundreds (in your case) of predictors is a sure sign that
it's your unsubstantiated wishful thinking, completely void of any
theoretical or empirical basis.


Quite the contrary, they are often trying to be "non-informative".
The last line is why its anti-Bayesian. That's the LAST thing a TRUE
Bayesian want his prior to be, "non-informative".

I don't know who is a True Bayesian, and I don't care.

That's rather obvious. Your statement is merely a redundant
expression
of your ignorance in the subject.

But by rejecting
non-informative hyperpriors, you are practically redefining bayesianism
as applied in _traditional_ statistics today. To understand this, see
some of the modern reviews (books), such as Bernardo and Smith, Gelman
et al., Gilks et al.

There does NOT EXIST any true Bayesian in the world in APPLIED
statistics who has to deal with more than 2 or 3 parameters because
NONE of them know how to solicite and represent their priors. They
only use the pseudo-Bayesian ideas to fool themselves and others
like yourself.


Even in the presence of good application-specific knowledge, there are
many perfectly valid reasons for using non-informative priors, part of
them technical and part procedural. You should know this if you have
ever used even a simple linear model.

Just don't PRETENT to be a Bayesian, if your forte is the use of
noninformative priors. Just try to learn the theory and method of
tranditional nonBaysian and Data Analytic techniques that, as
imperfect as they are, are infinitely more understandable and
respectable than being a Bayesian Quack -- which takes the WORST
of both worlds -- Bayesian and non-Bayesian.


One could also ask: why would you use a linear model if you are the
kind of True Bayesian you claim to be? Surely a linear model never
presents your prior knowledge accurately. Linear models are an
artificial subset of a much larger model family - by restricting
yourself to linearity without any deviations, you are essentially using
a highly artificial prior on that larger model family, and that surely
does not accurately present your prior knowledge. Are you excusing on
the basis of technical reasons, laziness or what? Why don't you put
your real prior information into your model?

That is why I don't PRETEND to be a Bayesian in my analysis of
linear models because it has its valid and useful techniques
WITHOUT the use of prior information which are NOT expressible
to reflect the true prior beliefs.

(Well, I guess you have rejected linear models long ago.)

You step into this neighborhood completely UNARMED in your
knowledge about any form of statistics, and completely unaware of
the thosands of articles I have posted since early 2005 in this
group about the PROPER use of linear models and model-building
methods -- that are obviously far beyond the level of your
statistical education.

You are NOT a Bayesian. I am one.

Fine with me. I have a problem and I want to solve it. If I have a
religion, it is nontechnical, or at least not related to formalisms for
handling uncertainty.

That's the characteristic of a Quack. Untrained in medical sciences,
he thinks he can cure all ills. A statistical Quack is of the same
ilk
I see your kind everyday, in THIS forum.

http://jscs.statjournals.net/ARTICLES/v09n03.html

Thanks - although a lot has happened since late 70's.

You typed before you read the sentence below.

Ridge Regression has virtually disappeared in JASA and other
reputable statistical journals of Statistics.

One could also say that it lives in every hierarchical model, but
please do not start to argument about that.

Like Factor Analysis, Ridge Regression has lived its brief life in
the statistical literature (while there were still some hope that they
have something to deliver). Since their 15 months (or years) of
fame, they are not completely abandoned by anyone worth his
salt in the field of Statistics. I have heard (from other editors)
that there was a moratorium (though unofficial) on the publication
of ALL Factor Analysis papers in JASA. That was in the decade
when I was an Associate Editor of JASA, in the 1970s to early
1980s. That moratorim seems to be in effect ever since, and so
is the application of Ridge Regression for anything!


NetFlix is NOT statistics. It's dataset is for exhaustive search and
ID based on a very special data that is only used BY NetFlix. It has
no use, nor is it usable by, or useful to, the field of statistics.

Again, I don't want to discuss semantics on such a general level here
and right now.

There are ways to handle such datasets that use statistical and
bayesian techniques and formalism. They exist, and are rigorous, and
effective, no matter how you or me want to call them.

You are merely repeating your buzz words in which you have no
knowledge and understanding that they are NOT valid nor justifiable
statistical methods.


If someone asks a technical question here on a context that is not
familiar to you, it would help this group a lot if you would shut up
instead of attacking the questioneer.

I have, in my detailed response to your uninformed and uneducated
allegations, GIVEN you and everyone else the context of those
areas that they are VERY familiar to me -- which is why I could
reject them as Quackery!

If you can't keep quiet, at least
reply with substance, not with an attack.

I replied with substance that you didn't even RECOGNIZE. That is
how deficient you are, in those areas in which you are just throwing
around a few words you read from other Quacks.

This used to be a relatively
civilized group with high signal-to-noise ratio, but now the situation
seems surprisingly poor - so poor that a moderated group is starting to
look attracting. You could, personally, help a lot by practicing a bit
of moderation on yourself.

Look at the history of Richard Ulrich's Quackery from 1995 through 2004
-- whose ERRORS and malpractices were not corrected or challenged
by ANYONE in this group.

It's high signal-to-noise alright, if you consider Quackery-Signal as
signal.

There has been plenty of Statistical Signals (hundreds of them on
my correction of Richard Ulrich's ERRORS alone -- they are ALL in
the sci.stat.math). If you know how to use the Google archives,
you can look up ANY statistical term in the advanced search of
group.google.com, together with author "Reef Fish", you WILL find
plenty of statistical signals, among also plenty of statistical NOISE,
by Richard Ulrich, Anon Bob O'Hara, Luis A. Afonso, Greg Heath,
and another handful of error-making NOISE-makers.
--
JS

Go peddle your statistical Quackery in some other groups. YOu'll
find company in the groups Illywacker peddles his statistics by
physicists who call themself Bayesians. You can also find 100%
of your signal to no noise in the new group scistatmath formed
by beliavsky because he didn't like the NOISE in this group.

The scistatmath group is where you belong.

Well, I just noticed your post of your OP in THIS thread there,
on Dec 15 -- the only post in that group since beliavsky's post
of "distribution of things" (now THAT's garbage is I ever seen
one in a statistical group).

Why are you back HERE making YOUR noise? Aren't you
perfectly happy with all the help you got in that group with
absolutely no signal and no noise?

You've had your opportunity to EXPOSE your own Quackery
while making your NOISE.

You should continue your subject in some physics/software/
groups that specialize in the malpratice of statistics -- you'll
find lots of signal of the kind you like to hear, and will be very
happy there.

Your continued presence in this thread in this group merely
confirmed what I said in my reply to you:

RF> I do feel obliged to reply for the reason that YOU are emotionally
RF> and intellectually immature and completely uninformed in those
RF> Statistical topics related to your problem that you are unable to
RF> take the VALID criticisms of your blind data dredging and misuse
RF> of Bayesian statistics and Bayesian ideas as if you're doing
RF> something valid in statistics.

Your free consulting session is OVER. I think Richard Ulrich is
out there eagerly waiting for you to contact him to be your paid
consultant, because he had all kinds of advice for you on NetFlix
and he must be your IDOL in this group because he had been
the most profilic, made most errors, and made most NOISE in
his years in sci.stat.math.

May you two find true happiness embracing each other and
blissfully share your Quackeries.

-- Reef Fish Bob.

.

Relevant Pages

  • Re: Beyond simple penalized regression
    ... are NOT to be taken likely by a true Bayesian. ... and therefore in a sense equipped with poor prior ... bayesian framework can be used _internally_ ... contemporary problem in statistics, sociology, epidemiology, ...
    (sci.stat.math)
  • Re: Beyond simple penalized regression
    ... OP> regression coefficients, ... Neither is even remotely Bayesian or statistical! ... OP> accuracy of the prior. ... in traditional statistics (with less automation and more ...
    (sci.stat.math)
  • Re: Beyond simple penalized regression
    ... OP> regression coefficients, ... Neither is even remotely Bayesian or statistical! ... OP> accuracy of the prior. ... in traditional statistics (with less automation and more ...
    (sci.stat.math)
  • Re: Beyond simple penalized regression
    ... and therefore in a sense equipped with poor prior ... bayesian framework can be used _internally_ ... NOT knowing what each and everyone of all the predictors used in the ... regression is a 100% unmistakable indication that you DON'T KNOW ...
    (sci.stat.math)
  • Re: Beyond simple penalized regression
    ... became feasible to use something more flexible than a conjugate prior. ... Since WHEN is a LOSS function part of a Bayesian inference? ... Most of you are familiar with ridge regression. ... pocketful of Mathematistry ideas. ...
    (sci.stat.math)
Loading