Re: Linford's Fallacy


"Tim Little" <tim@xxxxxxxxxxxxxxxxxx> wrote in message
news:slrnhcoar1.7rs.tim@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
On 2009-10-07, Jon Slaughter <Jon_Slaughter@xxxxxxxxxxx> wrote:
In fact to be more related to the idea of entropy we would probably want
to
define this as something more resonable(something symmetric):

sum(ln(1 + |A_k - A_(k+1)|))

This way the entropy of the sequence of all heads or all tails is 0. All
other sequences have larger entropy.

If I had to define entropy of a single sequence I'd be more inclined
to use a definition based around Kolmogorov complexity, or something
like it. The sequence of all zeroes has minimal entropy in that case
as it is produced by a quite trivial algorithm. The sequence of all
ones is also quite trivial.

However, it appears to me that entropy is best defined over
probability distributions, not sequences.


The sequences with alternating elements(010101... and 1010101...)
are the ones with the most "entropy".

That suggests that it is a poor choice of measure for something called
"entropy", which generally denotes some form of "unpredictability".



My only point was to show that one could create a definition to show that
some sequences are more "special" than others. I used the term entropy
because generally we tend to think pseudo random looking sequences(even if
they are not) are more complex... hence have a higher entropy. The sequences
of identical elements obviously then have the lowest entropy. Hence my
"entropy" function. It obviously only proves the point but may not have any
direct relation to standardized entropy functions.

I would agree with Kolmogorov's complexity idea. I just don't know much, if
anything, about complexity theory. What I do know is that some strings are
more complex than others. Seems like Kolmogorov has found a mathematical
way to state this precisely.

In fact what the original posters issue is, is exactly with the complexity
issue. The quoted person is clearly categorizing the sequences in terms of
complexity and showing that the least complex sequences have the lowest
probability. While all sequences have the same probability of being samples,
the complex sequences have a higher probability of being chosen than the
simple sequences.

I imagine we can make the original argument very precise using Kolmogorov's
complexity theoy with probability theory.




.

Quantcast