Re: transformation of regressors to remove collinearity

On Mar 11, 10:08 am, papu <prac...@xxxxxxxxx> wrote:
Actually because of model governance rules we are
required to use a certain minimum number of variables. The VIFs need
to be less than 3 and correlation coeffcient should be less than 0.7.
I have some predictors that have correlation coefficient 0.75 or
so..they are not highly correlated but not desirable either..

If I were you, I would do the following (to make the long story short)

1. The first way (possibly the best in my opinion) is to combine/
delete the correlated
variables based on previous results or theory. For example, the track
width
and wheelbase of a vehicle can be combined into planview.
(planview=trackwidth*wheelbase). This approach may result in a model
with very few predictors.

Hence I
thought why not separate the correlated and uncorrelated parts between
the predictors using geometric decomposition or projections and use
those transformation as predictors. This way the collinearity is zero.
But the question remains how to decompose these predictors and will
the predictive power of the transformed predictors remain the same as
a group.- Hide quoted text -

2. Apply PCA (principal component analysis) and then PCR (principal
component
regression) or PLS (partial least square). All of them are geometric
projections/decomposition of the predictors. They are "mere"
mathmatical transformation of predictors and they provide
correlational structure among predictors (hopely).

By the way, if your data are from observational studies (not from
"controlled" experiments where the predictors are orthogonal, samples
are balanced, confounding variables are controlled), the
interpretation should be very careful (almost futile
in my experience). As you already know, prediction and interpretation
are completely seperate issues.

Hope this helps.

Sangdon Lee, Ph.D.,
GM Tech. Center





.

Relevant Pages

  • Re: Principal Component Analysis
    ... That set can be all the raw variables or the set of summative scales based of the factor analysis. ... PCA and "Correlation among predictors". ... prediction with no fear of loosing information, ...
    (sci.stat.consult)
  • Re: Questions about square errors
    ... Take a look at the 10X10 correlation coefficient matrix and the ... multicollinearities. ... least squares and/or multiple regression. ... Your model may have several unnecessary predictors. ...
    (sci.stat.math)
  • Re: Principal Component Analysis
    ... When you have 30 input variables, and they are correlated, ... predictors are correlated and I am fully in agreement with his ... Interpretation assumes cause-and-effect relationship. ... repeated so many times: Correlation is not causation. ...
    (sci.stat.consult)
  • Re: transformation of regressors to remove collinearity
    ... Then the correlation between those ... projections we can get a regression equation without collinearity. ... It seems a *little* bit fruitful if all the useful Predictors ...
    (sci.stat.math)
  • Re: transformation of regressors to remove collinearity
    ... angle between the two predictors. ... Then the correlation between those ... projections we can get a regression equation without collinearity. ... Fit a regression of each pair of predictor variables with each other. ...
    (sci.stat.math)