Re: Zipf law and programming bias?

Maarten van Reeuwijk,
A lot of these scaling as log-log
have a high and low end tailing effect
in my experience.
What that says is the data for very large occurances
and / or very small
flattens out.
We chose a range in the middle that is linear.
As Stewart Robert Hinsley in another thread pointed out
four powers of the scale isn't very self-similar compared to theoretical systems with greater than 7 powers of self-similarity.

What we do know is that material with more "information"
has a different entropy than
that with less.
The problem is real world measures of some observation at very small scales or very large are not always very accurate.
Like getting the age of the universe from Hubble
expansion data...
it changes everything they get a better telescope ... usually upward.
Right now there isn't a lot of control on how Zipf law is used
by experimenters and how they interpret their results.
I've posted here one on Pollack's paintings before:
http://www.aip.org/pnu/1999/split/pnu432-4.htm

Where it started was in English language texts:
Word frequency in Zipf law of Shakespeare's blank verse:data here
http://www.mta75.org/curriculum/english/Shakes/hamfreq.html
http://en.wikipedia.org/wiki/Zipf's_law
The idea is that poetry constrained by some rule like blank verse has a different information content and thus, slope of the line.

Zipf law plots is also used to study "animal language" by plotting frequencies of sounds or in the case of sign language actions.

It is now a wide spread statistical tool,
but rules for it's uniform use are really well established?
Maarten van Reeuwijk wrote:
Roger Bagula wrote:


Roger Bagula wrote:

http://www.sigmaxi.org/meetings/archive/forum.2001.prog.pap.shtml


Taste in music does appear to be "quantifiable"
in terms of the kind of information that the music contains.


It is an interesting concept and a good idea to use MIDI files to analyze
the music. Their work is reported in this article in the Proceedings of the
2002 IEEE Southeastern Conference:

http://stono.cofc.edu/~mccauley/TReports/2001-7-2.pdf

The evidence is not too convincing, as without knowledge of the -1 scaling
of Zipf's law I would have drawn very different "best fits" to these
curves.

Maarten

.