Re: Information theory

Carlos Moreno wrote:

>> the only definition i can think of in an information theoretic sense is
>> the definition for entropy
>
> Err, no. Entropy is the average self-information of a
> source process (the average information of the possible
> outcomes of the process) -- notice that the concepts apply
> to two different things: information applies to an event,
> and entropy applies to a process that generates events.

There are lots of definitions of entropy.
Algorithmic entropy is a perfectly valid (and well-defined) use of the word.

You use the word in a different sense.
That's life.

--
Timothy Murphy
e-mail (<80k only): tim /at/ birdsnest.maths.tcd.ie
tel: +353-86-2336090, +353-1-2842366
s-mail: School of Mathematics, Trinity College, Dublin 2, Ireland
.

Relevant Pages

  • Wikipedia "random number"
    ... randomness, in the information theoretic sense of entropy is required. ... for insufficient entropy has often ...
    (sci.math)
  • Re: Information theory
    ... >> the only definition i can think of in an information theoretic sense ... the entropy His the number of bits necessary to ... but six possible outcomes for the die toss. ...
    (sci.math)
  • Re: Information theory
    ... > Err, no. ... if you're going to disagree with something i say, at least include the entire sentence. ... if you would have read my entire post, you'd see that i defined entropy correctly, as an alternate for a definition for information, NOT as a definition for information itself. ...
    (sci.math)
  • Re: Wikipedia "random number"
    ... randomness, in the information theoretic sense of entropy is required. ... The statement "They are statistically independent in that no ...
    (sci.math)
  • Re: Wikipedia "random number"
    ... in the information theoretic sense of entropy is required. ... Consider the mathematical constant p, 3.14159265358979..., which known ... some purposes requiring randomness in the sense of entropy. ...
    (sci.math)