Re: where do so many tenses come from?

Joachim Pense wrote:

Information theory gives the standard definition of "information". The
information of a sign (in this case a phoneme) within a sequence of signs
is the negative dyadic logarithm of the probability that this sign occurs
in this position.

You can describe it as the number of bits you need to encode it if your
coding system is optimized to produce optimally short bit strings for the
data source (=language in our case).

So if there are less signs to choose from, then for each the probability to
occur in any given position is higher (on average), so the information is
less.

I guess this definition encloses both syntagmatics and paradigmatics.

But that is oblivious to the contrastive nature of elements and seems orthogonal to the question. It treats {1, 2, 3, 14} and {0, 90, 180, 270} equally. It knows nothing of the actual informative value of phonemes.

What informative value should phonemes have apart from the fact that they
are different?

Contrast. In everyday terms, {s S T z Z D} is typically a less informative set than {r t v s g m}. Their relative value depends on the language, of course.

However, I would suppose that if the phoneme set is smaller, the amount of phonemes needed per morpheme is bigger if there are to be enough morphemes (the reverse side of what you describe above), so each phoneme could be seen as carrying a smaller part of the 'information'.

Isn't that exactly what I said? if the phoneme set is smaller, then each
carries less information.

The result may be the same (I'm not sure), but the reason is different. It only becomes similar if you bring in zip's dictionary size. It's not that each phoneme carries less information becuase the average probability is higher, but that the amount of phonemes needed to say something increases.
--
am

laurus : rhodophyta : brezoneg : smalltalk : stargate
.

Relevant Pages

  • Re: where do so many tenses come from?
    ... with a chance of 1/512. ... Then, on average, each phoneme contains: ... The average amount of information per character is meaningless, ... phoneme occurs with equal probability. ...
    (sci.lang)
  • Re: where do so many tenses come from?
    ... information of a sign (in this case a phoneme) within a sequence of signs ... So if there are less signs to choose from, then for each the probability to ... But that is oblivious to the contrastive nature of elements and seems ... informative set than. ...
    (sci.lang)
  • Re: where do so many tenses come from?
    ... Joachim> content of each phoneme you encounter in a text is less ... The average amount of information per character is meaningless, ... phoneme occurs with equal probability. ...
    (sci.lang)
  • Re: where do so many tenses come from?
    ... information of a sign (in this case a phoneme) within a sequence of signs ... is the negative dyadic logarithm of the probability that this sign occurs ... So if there are less signs to choose from, then for each the probability to ... of phonemes needed per morpheme is bigger if there are to be enough ...
    (sci.lang)
  • Re: where do so many tenses come from?
    ... So if there are less signs to choose from, then for each the probability to ... However, I would suppose that if the phoneme set is smaller, the amount of phonemes needed per morpheme is bigger if there are to be enough morphemes, so each phoneme could be seen as carrying a smaller part of the 'information'. ... (Notice that if some many-phonemed language happens to have large morphemes one's back to square 1.) ...
    (sci.lang)