Re: where do so many tenses come from?

In article <8764lmtza7.fsf@xxxxxxxxxxxxxxxxxxxxxxxxxx>,
Lee Sau Dan <danlee@xxxxxxxxxxxxxxxxxxxxxxxxxx> wrote:
"Herman" == Herman Rubin <hrubin@xxxxxxxxxxxxxxxxxxxx> writes:

>> Now, consider language A with a phoneme set of size 65, where
>> the first phoneme occurs with a probability of 7/8, and the
>> rest occurs with a chance of 1/512. Then, on average, each
>> phoneme contains:

>> {-log_2 (7/8) + -log_2 (1/512) * 64 } / 65 ={3-log_2 7 + 9*64}
>> / 65 = 8.8645...

>> bits of information.

Herman> This is wrong. If the probabilities are p_i, the
Herman> information in the language is the sum of - p_i *
Herman> log_(p_i).

That's the formula for entropy, not information.

Information has been called negative entropy.


Herman> The average amount of information per character is
Herman> meaningless,

I know. But since someone wants to make claims on it, I let him see
that claim is invalid.


Herman> and it is the total information which matters.

Of course!


Herman> So the amount of information is -7/8*log_2(7/8) + 9*64/512
Herman> which is approximately 1.3176 bits of information.

Isn't that that the entropy?




--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hrubin@xxxxxxxxxxxxxxx Phone: (765)494-6054 FAX: (765)494-0558
.

Relevant Pages

  • Re: Oy, meh hayah lanu
    ... >And that "thought" is a function of the language you speak and culture ... are those of the Statistics Department or of Purdue University. ... Herman Rubin, Department of Statistics, Purdue University ...
    (soc.culture.jewish.moderated)
  • Re: DH Idiocy 1
    ... discovery of quantum uncertainty. ... are those of the Statistics Department or of Purdue University. ... Herman Rubin, Department of Statistics, Purdue University ...
    (soc.culture.jewish.moderated)
  • Re: New mikveh wars in Israel
    ... reasoning makes a major use of the law of the excluded ... are those of the Statistics Department or of Purdue University. ... Herman Rubin, Department of Statistics, Purdue University ...
    (soc.culture.jewish.moderated)
  • Re: The Great Reverse Schlep Tour
    ... And the Protestants hate the Catholics ... are those of the Statistics Department or of Purdue University. ... Herman Rubin, Department of Statistics, Purdue University ...
    (soc.culture.jewish.moderated)
  • Re: Matrix question
    ... indicate the Kronecker product. ... I do not claim that these views are those of the Statistics Department or of Purdue University. ... Herman Rubin, Department of Statistics, Purdue University ...
    (sci.stat.math)