Re: an true information theory

From: Stephen Harris (cyberguard1048-usenet_at_yahoo.com)
Date: 02/27/05 

Date: Sun, 27 Feb 2005 22:14:12 GMT

"John Wilkins" <johnSPAM@wilkins.id.au> wrote in message
news:1gsnh6d.umidkz1ijytkaN%johnSPAM@wilkins.id.au...
> Stephen Harris <cyberguard1048-usenet@yahoo.com> wrote:
>
>> "John Wilkins" <johnSPAM@wilkins.id.au> wrote in message
>> news:1gsdj78.19ztr1a1xv4r7sN%johnSPAM@wilkins.id.au...
>> > <examachine@gmail.com> wrote:
>> >
>> >> John Wilkins wrote:
>> >> > borges2003xx@yahoo.it <borges2003xx@yahoo.it> wrote:
>> >> >
>> >> > > But is there someone who has developed or is interesting into
>> >> > > developing a theory in which deals with structure, pattern and
>> >> > > randomness in common sense ??
>> >> > >
>> >> > Not a mathematical theory. There are any number of semiotic or
>> >> > teleosemantic theories of information. None of them amount to >>
>> >> > much more
>> >> > than stating the ordinary view of information in technical terms,
>> >> > although semiotics can at times be quite clarifying.
>> >>
>>
>> I think the following approach is mathematical and it appears to be
>> part of a larger idea that includes semiotics:
>>
>> http://www.cerfacs.fr/algor/Qualitative/Qualitative.html
>
> Thanks. I'll read it through.
>>

I don't understand what missing concept is being talked about, that
such a concept is missing:
"structure, pattern, and randomness in common sense".

You apparently understand what he means and agree with him.
What is involved that is not included in an algorithmic approach
to constructing a modern AI program? Do you mean a
mathematical theory of consciousness? What does that have to do
with Algorithmic Infomation Theory? I've read at least two dozen
papers on AIT and cant' tell what the two of you are talking about.
Doesn't the probability theory approach of Bayes reasoning to AI
include or is "structure, pattern, and randomness in common sense"?
"Rational Agents in Probabilistic Environments"

There is also an AIT approach to Artificial Intelligence:
http://www.hutter1.de/ "Universal Artificial Intelligence"
Sequential Decisions based on Algorithmic Probability by Marcus Hutter

"This book presents sequential decision theory from a novel algorithmic
information theory perspective. While the former is suited for active agents
in known environments, the latter is suited for passive prediction in
unknown environments.

The book introduces these two well-known but very different ideas
and removes the limitations by unifying them to one parameter-free
theory of an optimal reinforcement learning agent embedded in an arbitrary
unknown environment. Most if not all AI problems can easily
be formulated within this theory, which reduces the conceptual problems to
pure computational ones. Considered problem classes include sequence
prediction, strategic games, function minimization,
reinforcement and supervised learning. The discussion includes formal
definitions of intelligence order relations, the horizon problem and
relations to other approaches to AI.

Keywords: Artificial intelligence; algorithmic probability; sequential
decision theory; Solomonoff induction; Kolmogorov complexity;
Bayes mixture distributions; reinforcement learning; universal
sequence prediction; tight loss and error bounds; universal Levin
search; strategic games; function minimization; supervised learning;
adaptive control theory; rational agents; exploration versus
exploitation."

"Sequential decision theory formally solves the problem of
rational agents in uncertain worlds if the true environmental
probability distribution is known. Solomonoff's theory of
universal induction formally solves the problem of sequence
prediction for unknown distribution. With a formal solution I
mean a rigorous mathematically definition, uniquely specifying
the solution. I unified both theories and gave strong arguments
that the resulting universal AIXI model behaves optimally in
any computable environment. I also made some progress towards
a computable AI theory. I constructed an algorithm AIXItl,
which is superior to any other time t and space l bounded agent.
The computation time of AIXItl is of the order t·2^l. The
constant 2^l is still too large to allow a direct implementation.
My main focus is on theoretical studies related to the AIXI model
and on further reducing the time-complexity of AIXItl down to a
point where it runs on todays computers with reasonable
computation time. This apporach may be characterized as a
*mathematical top-down approach to AI.*" [SH: *'s are mine]
http://www.hutter1.de/ai/selfopt.htm

SH: I think there is a well-founded mathematical theory for
"Rational Agents in Probabilistic Environments"; although such
programs have not been successfully practically implemented,
I think it qualifies as a 'developed theory'.

Regards,
Stephen