Re: large negative parameter correlations in regression


Stephen Clark wrote:
I have a simple 3 variable plus intercept regression model with good
goodness of fit statistics - high Rsq and t-ratios.

When you say you have "simple 3 variable plus intercept" regression
model, do you mean just a multiple regression of Y on three X's with
an intercept, or do you mean something else?


When I calculate the
correlation matrix of the parameter estimates I get a large negative
correlation between the intercept and one of the other parameters (-0.98).

Did the correlations come from the Covariance matrix of the estimated
betas? That is, is your correlation matrix derived from

inverse (X'X) * s^2

where X includes the column of 1's and your three X variables, and
s^2 is the variance of the residuals?

That would be the Covariance matrix of your estimated (bo, b1, b2, and
b3).

If you negative correlation -.98 came from that Cov matrix, then it
would mean one of your estimate bi is negatively correlated with
the estimate of bo. It is related to the multicollinearity concept
but not a measure of multicollinearity.


To investigate whether I have multicollinearity I have used the R-package
and the VIF is only given for the non-intercept parameters in the model (the
VIFs are 1, 7.5 and 7.5).

Leave the packages out of any such discussion because unless you
can describe exactly what the package computes and delivers, one
can never be sure what you are talking about.

Typically VIF refers to the reciprocal of (1-R^2) where R is the
multiple R of one X regressed on the remaining X's in the regression.

If so, then your VIF simply means that the max R between any one
of the X's with the other 2 is at most .93. hardly an alarming factor.


There are no worrysome correlations in the ACTUAL
variables, including a constant intercept.

What do you mean "worrysome correlations in the actual variables"?
:-)

How do you know? Where are your partial correlation information?
(That is in part imbedded in the significance of the T's)

Should I be concerned? Thanks.

My concern is your throwing these terms around without apparent
theoretical understanding of what they are telling you. "RULES OF
THUMB" in any regression analysis are dangerous toys to play
with. It is easy to become habit forming and become all thumb.

I would suggest you pay little attention to what Bob O'Hara says in
Regression matters. He is full of thumbs and a frequent Quack on
that subject.

-- Reef Fish Bob.

.

Relevant Pages

  • Re: Problem related to a linear regression
    ... iteratively refine this function F until you get the best correlation ... In your Test_2.gif I assume that the pale blue line is ... in the first regression you minimised squared deviations in the y- ... and in the second you minimised squared deviations in the x- ...
    (sci.math)
  • Re: r-Squared Question
    ... to wit: The correlation between Y and Yhat ... > My point was that while, as you say, "For OLS fitted regression, R^2 is ... interpretation, as defined by RegSS/TotSS. ...
    (sci.stat.math)
  • Re: A basic question on Canonical Correlation Analysis
    ... > Reduced-rank Regression, and I really did not mean Principal-Component ... The Canadian Journal of Statistics 8: ... of the problem of Canonical Correlation Analysis. ... > inputs, find its least principal component, then project each sample ...
    (sci.stat.math)
  • Re: significance of correlation affected by sample size?
    ... >> value OR significance of ANY correlation coefficient! ... >> USELESS result in a regression problem. ... handed out in said conference). ...
    (sci.stat.edu)
  • Re: Problem related to a linear regression
    ... The best correlation is found by minimising ... the iterations but *before* doing the regression between Xf and Y. The ... the ODR already in the iterative part? ...
    (sci.math)