Re: a question about mutual information

From: Roger L. Bagula (rlbtftn_at_netscape.net)
Date: 08/25/04 

Date: Wed, 25 Aug 2004 19:37:04 GMT

The Shannon information entropy of a Jacobian matrix M is:
( page 91 of The Mathematical Theory of Communication)
Det[M]=d
H=Ho+Log[d]
It is my guess that this is the kind of relationship that you are
looking for.
Since you are using a Gausian random matrix in your question,
it seems wise to ask if you are thinking in terms of the expectation
value of an infinite number of such random matrices or not?
The determinant of a single random matrix can have almost any value from
singular to infinity, but the expectation of the average of such
matrices can be calculated as Shannon has shown in his book. ( page 92):
H=W*log(2*Pi*e*N)
N is the white noise power and W is the band width.
roy wrote:
> Hi,
>
> (1)Assume X = R S where R is a random matrix with i.i.d. Gaussian
> entries, S is a constant matrix. Given X , R, and S, how to compute
> the entropy of X, i.e. H(X)= ?
>
> (2)and the mutual information between X and S is
> I(X;S) = H(X) - H(X|S)
> = H(X) - H(RS|S)
> = H(X)
> Is it correct?
>
> Thank you very much 

-- 
Respectfully, Roger L. Bagula
tftn@earthlink.net, 11759Waterhill Road, Lakeside,Ca 92040-2905,tel: 
619-5610814 :
URL :  http://home.earthlink.net/~tftn
URL :  http://victorian.fortunecity.com/carmelita/435/