Re: Using Ridge Regression to disentangle highly correlated explanatory variables

In article <fa46df71-e64b-4ee5-93a2-55d4dc605744@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>,
Old Mac User <chendrixstats@xxxxxxxxx> wrote:
On Mar 13, 2:25 pm, JohnF <jf...@xxxxxxxxxxx> wrote:
Folks,

Need your advice and any practical solutions.

We recently conducted a retrospective regression analysis where 3
variables were highly correlated (high VIFs). Decided to use a
principal components approach to create a factor score for input into
the regression model, which did it's job at reducing the VIF greatly.

However, the three highly correlated variables were each of great
interest. A colleague suggested using Ridge Regression to disentangle
the relative impact of each of the three explanatory variables. This
did show that one of the three variables was much more impactful.

Now I'm left wondering if this makes sense, given they were so highly
correlated to begin with. Wouldn't we conclude that they are all
equally contributing - i..e, the factor loading can be divided in
terms of relative impact equally among the three variables?

What's your opinion on this type of issue. I need some practical
advice, point of view, and/or alternate approach to consider.
Remember that the three variables are each of particular interest, so
need to somehow cull out their relative impact.

Very much appreciate any and all help. Thanks!

John

The fact that they are highly correlated means (among other things)
that you may as well pick one variable to use as a predictor and
ignore the others. However, recognize that in doing so you are
assuming that those high correlations are "permanent and enduring" and
will not disappear in future data. In other words, those high
correlations are structurally associate, and not just due to chance.

When all is said and one, there's really nothing you can do to get
valid estimates of the effects of those three variables separately
from one another. My suggestion is to forget about ridge regression.
It's an endless swamp. OMU

If you are interested in the structure, to decide
policy changes, only something like ridge regression
can help you. All such models are somewhat Bayesian,
so prior information can be included instead of a
simple Ridge procedure.
--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hrubin@xxxxxxxxxxxxxxx Phone: (765)494-6054 FAX: (765)494-0558
.