Re: transformation of regressors to remove collinearity
- From: vontressms@xxxxxx
- Date: Tue, 11 Mar 2008 06:05:56 -0700 (PDT)
On Mar 10, 8:34 pm, papu <prac...@xxxxxxxxx> wrote:
Hi All,
In regression, is there any way we can transform the predictors to
remove the collinearity?
There is this article in wikipedia which talks about geometric
interpretation of correlation.http://en.wikipedia.org/wiki/Correlation#Geometric_Interpretation_of_...
Basically here they say that, if we look at the predictors as vectors
then the correlation coefficient can be treated as the cosine of the
angle between the two predictors.
Lets say we define our dimensions first take projections of predictors
(vectors) on to those dimension. Then the correlation between those
projections will be zero because they lie on the dimension
(correlation between dimensions is zero). If we make use of these
projections we can get a regression equation without collinearity.
Well that is the notion I have and I am not able to find any material
on that. Can somebody point me to information on this?
Or if this is a pretty obvious fact already considered in regression,
please let me know how.
-Papu
Papu,
Collinearity is another way of saying that two or more predictors
are redundant. You really only need one since the others provide the
same information. There are lots of ways to deal with this problem,
but I will share a simple way that works for small numbers of
predictor variables.
Fit a regression of each pair of predictor variables with each other.
Let the response (independent variable) be one predictor and the
"predictors" be another set of the predictor variables. If there is
high correllation between two predictor variables, then they are
supplying redundant information and one of them can be deleted. This
is the basic idea of all of the methods for removing collinearity, but
it delves into obsure words like singular valued decomposition or
principal components analysis - which are harder to interpret.
Also, note that a near constant variable will be collinear with the
intercept, so near constant variables can also be ignored.
Once you have removed all of the redundant predictors, you should be
able to fit your regression model without collinearity.
Mark
.
- Follow-Ups:
- References:
- Prev by Date: solutions manual of the title MODERN CONTROL SYSTEMS THEORY AND DESIGN by STANLEY M SHINNERS
- Next by Date: Re: transformation of regressors to remove collinearity
- Previous by thread: Re: transformation of regressors to remove collinearity
- Next by thread: Re: transformation of regressors to remove collinearity
- Index(es):
Relevant Pages
|
