Re: simple way to update the inverse of the covariance matrix with each data point


Gordon Sande wrote:
On 2006-09-29 12:21:15 -0300, Ken Butler <butler@xxxxxxxxxxxxxxxx> said:

On 29 Sep 2006 05:54:25 -0700, "Greg Heath" <heath@xxxxxxxxxxxxxxxx>
wrote:

Googling on "Sherman-Morrison" (thanks Ken & Reef) led
me to enough info to complete the derivation.

Derivation seems to be a bit strong. It is simple in scalars and then
all one really has to do is verify the simple algebra.

Try Golub & van Loan for matrix computations.

The other issue is do you really want to be manipualting the inverse?
See the works by Bjorck. Certainly his SIAM book on Least Squares. Or
the draft chapeters that he has on his web site.

Ken's original question of how to UPDATE an inverse covariance
matrix when an observation is added or delected.

For said purpose, it's highly desirable if doable in an efficient way.

(1) I even had a JASA paper years ago (1974) "Comparison of Several
Algorithms for Computing Sample Means and Variances," for the subject
which may be required for very large data sets, OR for updating those
simple quantities when DELETING an observation (such as in regression
diagnostic, to see what would happend) and ADDING the observation
back to the original. There were a number of commands in the
interactive package IDA for doing just that.

(2) That same is true for the "what if" analysis in regression of
delecting
and entire row of observations. The inverse of the covariance is the
heart of s^2 inverse(X'X) which has all the covariances for the
estimate
betas in the updated regression. The "what if" computation is much
easier for a adding or deleting a VARIABLE in the regression, and I
did it via Beaton's operator SWEEP. For deleting an entire ROW of
observations, I simply repeated the "delete observation" for each
variable, for the re-inversion of X'X because I don't know of a good
way to update the (X'X) inverse matrix.

Those are just two reasons why those capabilities should be in every
regression package. Most packages have the option to delete or
add a VARIABLE; but most have just the brute force recomputation
of everything when a single row or a single observation is changed
or deleted. This would be very wasteful for a regression with a few
thousand observations (rows), to have recompute everything if ONE
number is deleted (such as an extreme outlier).

-- Reef Fish Bob.

.

Relevant Pages