Re: where do so many tenses come from?

Am Mon, 03 Apr 2006 22:26:21 +0100 schrieb António Marques:

Joachim Pense wrote:

Information theory gives the standard definition of "information". The
information of a sign (in this case a phoneme) within a sequence of signs
is the negative dyadic logarithm of the probability that this sign occurs
in this position.

You can describe it as the number of bits you need to encode it if your
coding system is optimized to produce optimally short bit strings for the
data source (=language in our case).

So if there are less signs to choose from, then for each the probability to
occur in any given position is higher (on average), so the information is
less.

I guess this definition encloses both syntagmatics and paradigmatics.

But that is oblivious to the contrastive nature of elements and seems
orthogonal to the question. It treats {1, 2, 3, 14} and {0, 90, 180,
270} equally. It knows nothing of the actual informative value of phonemes.

What informative value should phonemes have apart from the fact that they
are different?

Contrast. In everyday terms, {s S T z Z D} is typically a less
informative set than {r t v s g m}. Their relative value depends on the
language, of course.

That's really an aspect I didn't take into account.




Isn't that exactly what I said? if the phoneme set is smaller, then each
carries less information.

The result may be the same (I'm not sure), but the reason is different.
It only becomes similar if you bring in zip's dictionary size. It's not
that each phoneme carries less information becuase the average
probability is higher, but that the amount of phonemes needed to say
something increases.

Yes, I think that's the same according to the given definition of
information.

Joachim
.

Relevant Pages

  • Re: where do so many tenses come from?
    ... with a chance of 1/512. ... Then, on average, each phoneme contains: ... The average amount of information per character is meaningless, ... phoneme occurs with equal probability. ...
    (sci.lang)
  • Re: where do so many tenses come from?
    ... So if there are less signs to choose from, then for each the probability to ... But that is oblivious to the contrastive nature of elements and seems ... of phonemes needed per morpheme is bigger if there are to be enough morphemes, so each phoneme could be seen as carrying a smaller part of the 'information'. ...
    (sci.lang)
  • Re: where do so many tenses come from?
    ... Joachim> content of each phoneme you encounter in a text is less ... The average amount of information per character is meaningless, ... phoneme occurs with equal probability. ...
    (sci.lang)
  • Re: where do so many tenses come from?
    ... So if there are less signs to choose from, then for each the probability to ... However, I would suppose that if the phoneme set is smaller, the amount of phonemes needed per morpheme is bigger if there are to be enough morphemes, so each phoneme could be seen as carrying a smaller part of the 'information'. ... (Notice that if some many-phonemed language happens to have large morphemes one's back to square 1.) ...
    (sci.lang)
Loading