Re: where do so many tenses come from?

"Joachim" == Joachim Pense <spam-collector@xxxxxxxxxxxxxxx> writes:

Joachim> Information theory gives the standard definition of
Joachim> "information". The information of a sign (in this case a
Joachim> phoneme) within a sequence of signs is the negative
Joachim> dyadic logarithm of the probability that this sign occurs
Joachim> in this position.

Well... you have to get these probabilities first.


Joachim> You can describe it as the number of bits you need to
Joachim> encode it if your coding system is optimized to produce
Joachim> optimally short bit strings for the data source
Joachim> (=language in our case).

What a big IF!


Joachim> So if there are less signs to choose from, then for each
Joachim> the probability to occur in any given position is higher
Joachim> (on average), so the information is less.

On average? Does this average translate to an assumption that the
probabilities are more or less EVENLY DISTRIBUTED among every sign?
I'm afraid this assumption isn't that valid. Take English as an
example. Does the phoneme /T/ occur almost as frequently as /i/?


Joachim> I guess this definition encloses both syntagmatics and
Joachim> paradigmatics.



--
Lee Sau Dan 李守敦 ~{@nJX6X~}

E-mail: danlee@xxxxxxxxxxxxxxxxxxxxxxxxxx
Home page: http://www.informatik.uni-freiburg.de/~danlee
.

Relevant Pages

  • Re: where do so many tenses come from?
    ... Joachim> "information". ... Joachim> phoneme) within a sequence of signs is the negative ... Joachim> dyadic logarithm of the probability that this sign occurs ... Herman Rubin, Department of Statistics, Purdue University ...
    (sci.lang)
  • Re: where do so many tenses come from?
    ... Joachim> "information". ... Joachim> phoneme) within a sequence of signs is the negative ... Joachim> dyadic logarithm of the probability that this sign occurs ...
    (sci.lang)
  • Re: where do so many tenses come from?
    ... Joachim> "information". ... Joachim> phoneme) within a sequence of signs is the negative ... Joachim> dyadic logarithm of the probability that this sign occurs ... But phonemes frequencies are not normally distributed. ...
    (sci.lang)
  • Re: intrinsic advantage of Latin alphabet over bopomofo (for Chinese)??
    ... Joachim> It happens that a sequence of two Kanji stands for one ... The first two are pretty formal; the third is more common and casual. ...
    (sci.lang)