Re: an true information theory

From: Stephen Harris (cyberguard1048-usenet_at_yahoo.com)
Date: 02/21/05 

Date: Mon, 21 Feb 2005 22:15:12 GMT

<borges2003xx@yahoo.it> wrote in message
news:1108981072.758765.321290@c13g2000cwb.googlegroups.com...
> It very silly.
> Shannon information theory. Most informative sequences are what in
> which every symbols is completely independent from other. like a
> roulette.
> Structure exists in sequences ranging from human language and music to
> the genetic information encoded in our DNA, anf finally, in our
> computer programs.
> Structure exist also in zip file. the web is full of rules, patterns,
> structures, or or whatever you want to say.
> But Shannon says that meaning is beyond this theory.
> Kolmogorov, Chaitin, Wolfram and so on, consider a non random >sequence,
> a small program that generate pi, for example, which has no pattern, no
> structure. flat, junk.
> But is there someone who has developed or is interesting into
> developing a theory in which deals with structure, pattern and
> randomness in common sense ??
>
> thank
> borges2003xx@yahoo.it
>

The newer definition of information does not stress meaningful content. AI
is already interested in meaning within a statistically emergent mentality.

AI is interested in the meaningful content of information and distinguising
syntax from semantics. Douglas Hofstadter wrote two fine books that
cover background for this topic. "Godel, Escher and Bach" and
"Metamagical Themas" which has a chapter 26, Waking UP from the
Boolean Drea, or, Subcognition as Computation, which challenges the
idea of "cognition is computation". Hoftadter writes, ... "If I interpret
it more strictly as "cognition is an activity that can be achieved by a
program that shunts around meaning-carrying objects called symbols
in a complicated way", then I don't buy it."

"There seems to be a genuine conundrum about how mere matter
could possess _reference_."

SH: I think a more detailed question than information is: What is a
symbol and how and what level do symbols arise and their relationship
to meaning and understanding. What part of symbols is subcognitive
and do not express or represent symbols at the neural level? At a
physical level, Koch has interesting things to say which bring in
attractors.
Often "random" is used in the sense of 'deterministic but no predictable'.
Jim Crutchfield has worked on measuring complexity. Interesting pages:
http://www.idsia.ch/~marcus/ai/index.htm

It is pretty much agreed that using an evolutinary approach, such as
how humans evolved intelligence, or running the supposed Universe
creating CA proposed by Wolfram will fail due to time constraints.

Hofstadter's books go into analogy and pattern recognition--People
are always saying how babies can recognize their mother's face but
computers can't, but computers can do difficult algebra problems.
Well this face recognition issue is connected to seeing and grouping
typesets with different letter styles. You may have seen one of those
tests with the random squiggley lines with some seemingly recognizable
letters included. One has to verify the symbols in order to get approved
as a human not some malware email gathering bot. That is AI in a nutshell.
Telling that some weird variation of "a" is not some weird variation of "i".

> But is there someone who has developed or is interesting into
> developing a theory in which deals with structure, pattern and
> randomness in common sense ??
>

The answer is of course. Read Hofstater and Koch, "Quest for
Consciousness" and develop the background. Discussing

> But Shannon says that meaning is beyond this theory.
> Kolmogorov, Chaitin, Wolfram and so on, consider a non random >sequence,
> a small program that generate pi, for example, which has no pattern, no
> structure. flat, junk.

has put the cart way in front of the horse's tail. You are talking about
obscure complexity issues and how they may peripherally relate to AI.
Your post shows you are unaware of symbols and symbol manipulation
issues which are at the heart of AI. There is no such thing as a "true"
information theory. I am going to post the reason why, but not explain it
because I've already spent enought time on this post.

http://www.cis.udel.edu/~case/colt.html

John Case's COLT Page

Consider the problem of finding a rule for generating a sequence of numbers
such as 9, 61, 52, 63, 94, 46, 18, 1, 121, 441, ... . Here is a rule for
this sequence. First compute the squares of successive integers beginning
with 3, but, then, to generate the sequence, use, in place of these squares,
the squares each with its decimal digits written down in reverse order
(ignoring any lead zeros). N.B. This rule can be written as a formal
algorithm (or computer program). The problem of finding such rules gets
harder as the sequences to generate get more complicated than the one above.
Can the rule finding itself be done by some computer program? Interestingly,
it is mathematically proven that there can be no computer program which can
eventually find (synonym: learn) these (algorithmic) rules for all sequences
which have such rules!

Here is a different problem. Given pictures of animals each (correctly)
labeled as being or not being a picture of a bird, eventually find/learn an
(algorithmic) rule for deciding, of each of those (past, present, and
future) pictures, whether or not it depicts a bird. People seemingly solve
this problem in childhood as well as many other problems of learning
(subconscious) rules to predict membership in concepts (such as the concept
of bird). Many cognitive scientists seek to model all of cognition
(including concept learning) by computer program. It is, then, an
interesting, very hard problem to write a computer program which can learn
(algorithmic) rules to predict membership in all the concepts that people
can . It's so hard we don't know how to do it yet, and we don't know how to
to prove it can't be done.

SH: Humans have trouble at the 7+2 memory boundary. Borges' post is quite
insightful for a younger explorer.

Relevant Pages

  • Re: an true information theory
    ... If a message has meaning then it has structure and will show a pattern. ... if there is a pattern in the sequence of numbers. ... solve the halting problem, which is uncomputable. ...
    (sci.math)
  • Re: Undecidability in Physics
    ... we can calculate the behavior of any Turing machine under certain ... Here is a rule for this sequence. ... N.B. This rule can be written as a formal algorithm (or computer program). ... of the universe as mathematical functions, ...
    (sci.logic)
  • Re: Snowflakes with High CSI
    ... figures will be construed as proof that you cannot measure CSI. ... or not detect a bias in a pattern is the same thing IDists claim to be ... IOW, the sequence not only has to have a specific sequence, but that ... my zip code +4 would, of course, have meaning for *me*, but might not ...
    (talk.origins)
  • Re: an true information theory
    ... > this sequence. ... First compute the squares of successive integers ... > algorithm (or computer program). ... Neither can a brain. ...
    (sci.math)
  • Re: Godel proved maths inconsistent not incompleteness theorem
    ... Do you think that a proof created by running a computer program, ... IF it were a proof in the sense of a finite sequence of formulas, ... either an axiom from a recursive set of axioms or derived by a ...
    (sci.logic)