Re: Multiple regression coefficients

The order of variables does MATTER if the independent variables are
collinear, not independent, near singular, or not full rank. This is
called multicollinearity.

The least square estimator of Y=XB is B=inv(X'X)X'Y

The FUNDEMENTAL fact from linear algebra is that if independent
variables are collinear (not independent, near singular, or not full
rank), the inverse matrix (X'X) are NOT unique. In other words, there
are infinite number of solutions to compute the inverse matrix. Thus a
small change in input variables cause large change in the beta
regression coefficients (B).

The order of variables seems not matter if you perform linear
regression several times with the same data set but the order of
variables are changed. However if one of a sample is deleted or the
values of collinear variables are modified slightly, you will have
significantly DIFFERENT regression coefficients due to computing the
inverse matrix of X'X. Also, the sequential sum of square will be
totally different if the order of variables are changed due to
multicollinearity.

Check the variation inflation factor (VIF) or the condition number
which is a ratio of the largest to the smallest eigenvalue for
multicollinerity. Both should be 1.

Hope this helps.

Sangdon Lee, Ph.D.
GM Tech. Center, MI.


*** Startz wrote:
> On 13 Nov 2005 06:55:37 -0800, "statukey@xxxxxxxxx"
> <statukey@xxxxxxxxx> wrote:
>
> >I am attempting to gain a better understanding of the multiple
> >regression coefficients. I have understood that the order in which the
> >variables are presented in the model is critical. This is because each
> >successive beta weight represent the covariance (correlation) with the
> >dependent variable while partialing out the contribution of preceding
> >variables in the regression equation.
> >snip
> >
> >statukey
>
> The order of variables has no effect on the coefficients in a multiple
> regression.
>
> -*** Startz
> ----------------------
> Richard Startz RichardStartz@xxxxxxxxxxx
> Lundberg Startz Associates

.

Loading