Re: What's the physics meaning for fisher information matrix?


"Rusty" <john@xxxxxxxxxxxxxxxxxxxxxx> wrote in message
news:d8acjt$5mh$1@xxxxxxxxxxxxxxxxxxxxxx
>
> "QQ" <junciu@xxxxxxxxx> wrote in message
> news:1118342632.980065.142660@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> The fisher information matrix is defined here
>> http://mathworld.wolfram.com/FisherInformationMatrix.html
>>
>> usually for a distribution
>> var, mean etc all have some physical meaning
>> such as variance is kind of fluctuation.
>>
>> So what's about fisher information matrix?
>
> AIUI the Fisher information matrix for a parameter estimation problem is
> much the same thing as covariance matrix of the parameter estimates for
> maximum likelihood estimation.
> Under a large sample, or small Gaussian noise assumption the derivatives
> in the Fisher Matrix (as in mathworld), reduce to estimating the
> covariance matrix of the ML parameters.
>

Apols, should have said the Fisher matrix is the inverse of the parameter
covariance matrix. The differentiations d2L/dtheta1.dtheta2 applied to a
Gaussian multivariate pdf at the maximum likelihood point will provide the
curvatures at this point which is equivalent to the inverse covariance (for
gaussians). So
(i) if the parameter estimation is a linear function of the observations in
a small region about the ML point
(ii) the observation noise is iid gaussian
(iii) the observation standard deviation is sufficiently small

then the parameter estimate is Gaussian distributed with covariances
specified by the inverse Fisher matrix. This is also the Cramer Rao lower
bound. See a good book on parameter estimation.

rusty


.