Re: Using Ridge Regression to disentangle highly correlated explanatory variables

I agree 100%.

I think I'm one of the people who like to recommend PCA, PCR, and PLS
to overcome collinearity (with some good reason). In my experience,
the most frequently asked question in many organizations is "Which
variable is the most sensitive to the response? (, so that conflicting
requirements can be prioritized). If I'm not ready to answer this
question, I'm doomed. I encounter this sensitivity question daily and
you never ever say "That can't be done" to your boss (at least
according to Bill Lane "4 Things You Should Never Say to the CEO") so
my recommendation. (There are other logical reasons that are specific
to my cases though(^.^)).

The original posting (...somehow cull out their relative impact...) can be
answered by statistics if and only if the data are from experiments
(samples are randomly assigned, confounding factors are controlled,
predictors are orthogonal, etc). AND LASTLY, THE PREDICTORS ARE
ORTHOGONAL OR AT LEAST WELL-CONDITIONED. Even the data are from
experiments but the predictors are collinear, the sensitivity of
predictors can NOT be estimated robustly.

Let me emphasize one more time (even though many people already knew):
The simple truth is that if variables are collinear, the individual
sensitivity (weights) of predictors can NOT be determined robustly no
matter what statistical methods are applied (including RR, PCR, PLS,
etc). They are mere mathematical transformation. However, in my
experience, I hardly see a person who understands this.

THIS MISUNDERSTANDING IS UNIVERSAL. I mean UNIVERSAL.

I came to a conclusion that statistis (on observational data) failed
to convey the ill-effects of collinearity. Collinearity is treated as
one of the many subjects to learn in statistical text books and many
assumptions/options to check in software.

I've never seen engineers (whose life depend on eigenvalue
decomposition) who understand that collinearity is the same as ill-
conditioning in solving systems of simultaneous equations.

IMHO, statistics should emphasize (thus has failed):
*Difference between experimental vs. observation studies
*Causation and association
*Prediction vs. Interpretation (thus sensitive, collinearity)
and hopely
*Dynamic vs. static behavior.

SD Lee.
.

Relevant Pages

  • Re: Using Ridge Regression to disentangle highly correlated explanatory variables
    ... "Statistics is about communicating formation from one ... sensitivity question daily and you never ever say "That can't be done" ... THE PREDICTORS ARE ORTHOGONAL OR AT LEAST WELL- ... data) failed to convey the ill-effects of collinearity. ...
    (sci.stat.math)
  • Re: LINEST maximum number of predictor variables
    ... > If the predictors exhibit near collinearity, ... > predictors from the group of predictors that are nearly collinear. ... >> properties of LINEST solutions long before hitting that hard coded ...
    (microsoft.public.excel.worksheet.functions)
  • Re: transformation of regressors to remove collinearity
    ... angle between the two predictors. ... Then the correlation between those ... projections we can get a regression equation without collinearity. ... Fit a regression of each pair of predictor variables with each other. ...
    (sci.stat.math)
  • Re: transformation of regressors to remove collinearity
    ... Then the correlation between those ... projections we can get a regression equation without collinearity. ... It seems a *little* bit fruitful if all the useful Predictors ...
    (sci.stat.math)
  • Re: collinear interaction but not for predictors
    ... > the substantive utility of interpreting multilcollinear interaction. ... > I have two predictors that are highly correlated, ... You are asking whether the collinearity is increased by ... What sense are you trying to draw? ...
    (sci.stat.consult)