(Help find fallacy) Covariance(LS)=Covariance(WLS)

From: John Creighton (JohnCreighton__at_hotmail.com)
Date: 03/26/05

  • Next message: skeptic8000: "Estimating r^2 from F ratio" 
    Date: 25 Mar 2005 17:49:19 -0800

    I have shown the covariance of the LS estimate to be equal to the
    covariance of the weighted LS estimate. I know this cannot be the
    case. Can someone help me find the fallacy.

    Performing some algebra on the covariance of the estimate:

    R2^-1(R2^-1)^T=Q2=E[x hat – x][x hat – x]=R1^-1E[z bar – R1 x][z bar –
    R1 x] (R1^-1)^T =
    R1^-1 T_1 E[z – C x][z – C x] T¬_1^T (R1^-1)^T=
    R1^-1 T_1 Q¬_m T¬_1^T (R1^-1)^T= R1^-1 T_1 S¬_m S¬_m^T T¬_1^T
    (R1^-1)^T

    Where:
    E is the expectation operator
    x hat is the estimate
    x is the parameter being estimated
    R=T¬1_1 C (This would be the LS estimate for independent identically
    distributed noise with identity covariance)
    C is the data matrix
    Q_m is the covariance of the measurement
    S_m is the choleskey factorization of Q_m
    ^T denotes transpose
    T1 is an orthonormal transformation with
    T1=[T1_1
        T1_2]
    R is in upper triangular form (not that it is rellevent)

    So far so good? Any mistakes?
    Now we say that there exists a matrix T2_1 where T2_1 T2_1^T = I so
    that:
    R2^-1 T2_1= R1^-1 T_1 S¬_m Then we get
    R2^-1 T2_1 S_m^-1 = R1^-1 T_1 =>
    R2^-1 T2_1 S_m^-1 T_1^T= R1^-1 =>
    T2_1 S_m^-1 T_1^T R1= R2 =>
    T2_1 S_m^-1 C =R2 =>
    T2_1 R_m C =R2

    Where
    R_m is the choleskey factorization of the measurement noise
    R2 is the choeskey factorization of the estimate covariance
    T2_1 is chosen to but the R_m C in upper triangular form

    Now the fallicy is that the above result is the same expression for
    the choleskey factorization of the weighted LS estimate. Clearly the
    weighted LS estimate has a smaller error then the LS covariance. This
    is easy to show in the scaler case. Anyway can anyone see my fallacy?

  • Next message: skeptic8000: "Estimating r^2 from F ratio"