Re: A multiple regression stumper
- From: "Ray Koopman" <koopman@xxxxxx>
- Date: 29 Sep 2005 14:35:23 -0700
rick.deshon@xxxxxxxxx wrote:
> Hi All.
>
> I can't figure out the solution to what should be a fairly
> straightforward regression problem.
>
> Assume you have a set of variables (X) that you use to predict a single
> variable (Y) in a standard multiple regression model. X is nxp and Y
> is nx1.
>
> In this model, Y = Xb + e, where e is a nx1 vector of residuals.
>
> The OLS estimate of b is inv(X`X)*X'Y. Consider b to be a (px1) vector
> of optimal weights that minimize the variance of e.
>
> One way to examine the quality of the fitted regression is to compute
> R^2 (the coefficient of variation or determination). R2 = (b'*
> cov_XY)/var(Y) where cov_XY is a px1 vector of covariances between the
> columns of X with Y (cov_XY = (X'Y)/(n-1)) and var(Y) is the variance
> of the vector Y. Conceptually, R2 is the ratio of predictable variance
> in Y to total variance in Y.
>
> I would like to compute R2 for non-optimal sets of weights. What
> happens to R2 as you use less and less optimal weights?
>
> This would be simple under normal circumstances but I'd like to do it
> for a special case where you don't know Y. In other words, you have X,
> b, and you know the R2 for the optimal model. Further, using knowledge
> of X, b, and the optimal R2 you can compute the variance of Y so you
> know that quantity also.
>
> Is it possible to estimate R2 for non-optimal weights if you know b, X,
> and var(y)? The only missing quantity is cov_XY but b clearly has
> information on these missing covariances of X and Y. I have not been
> able to determine a unique solution to this apparently simple problem.
> Perhaps an orthogonal projection?
>
> Thanks for any insights you can provide!
>
> Rick
Given X, b, R^2.
y = X.b + e for some n-vector e that is orthogonal to the columns of X,
where "." denotes matrix multiplication. Then
var[y] = var[X.b] / R^2 = b'.cov[X,X].b / R^2
and cov[X,y] = cov[X,X].b .
For any arbitrary weight vector a ,
corr[X.a,y] = cov[X.a,y] / sqrt[ var[X.a] * var[y] ] ,
where cov[X.a,y] = a'.cov[X,y] = a'.cov[X,X].b
and var[X.a] = a'.cov[X,X].a .
.
- Follow-Ups:
- Re: A multiple regression stumper
- From: rick . deshon
- Re: A multiple regression stumper
- References:
- A multiple regression stumper
- From: rick . deshon
- A multiple regression stumper
- Prev by Date: Re: Solutions - Statistical Inference Casella & Berger
- Next by Date: Re: Solutions - Statistical Inference Casella & Berger
- Previous by thread: Re: A multiple regression stumper
- Next by thread: Re: A multiple regression stumper
- Index(es):
Relevant Pages
|
