Neural netss (was Re: death of the mind.)

From: Wolf Kirchmeir (wwolfkir_at_sympatico.ca)
Date: 08/27/04 

Date: Fri, 27 Aug 2004 09:46:51 -0400

Eray Ozkural exa wrote:

[...]
>
> Here is a question for you, Wolf. What does the "face recognition"
> circuitry in your brain accomplish? If it does not contain reference
> states, what does it contain then?

As I understand it, the "face recognition module" responds to specific
inputs from the otber areas of the VC/etc, which in turn respond to
inputs from the retina (which, as you know, organises the visual data
transmitted by the rods/cones). The "reference state" that you and
others refer to is as I understand it the organisation of the NNs in the
face recognition module. OK, now what? I see little utility in talking
about it that way. The abstract terminology may help if you want to
mimic the operation of the VC in a digital computer, since it suggests
architecture, but it doesn't help much otherwise that I can see.

The phrase may in fact mislead, since it implies a comparison (and the
comparison has been explicitly invoked by some contributors to this
thread.) But I don't see any comparison happening - there is AFAIK no
"comparator module" that takes input from the "perception module" and
the "face recognition module" and outputs "Yup, that's Johnny." The fact
is that information about edges, etc is produced in the retinal layer,
that this information passes to various NNs whose output goes to other
NNs, etc until the final output is "That's Johnny." At each layer, the
information is reorganised - that is, the inputs to the next layer have
a different pattern. Note that the face recognition module is not as
simple as the name impies - some people who have had damage in that area
can still tell they are looking at a face, they just can't tell who it
is. Others can't even tell they are looking at a face. Since the damage
is to the NN, the "reference state" can only be the organisation of the
NN, as I said above. Again, now what? The organisation of the NN
determines how it operates, is all. Calling it a reference state doesn't
help us understand how it's organised.

Sidebar: The closest computer analog I can see to what happens is bit
masking. However, in a computer, the bit mask is imported from storage -
there is no similar event in the brain that I can see. The NN itself
masks or filters the input; it doesn't import a mask.

The reason I keep talking about brain physiology as I understand it is
that I've never been happy with abstract models that ignore the messy
details. "Reference state" is such a concept, since it applies to
devices that function differently than NNs. Sure, it's very satisfying
to recognise some abstract identity (eg, abstractly considered, the
state of a network can be thought of as stored information), but the
messy details keep interfering with insight (eg, when the network
changes state, some or all of the previous information is lost, unlike
information stored by other means.) Abstract models are useful - they
obviously help in constructing devices that work-alike some observed
brain function/system behaviour/etc.

> If you can answer this question, then I have a harder followup. What
> do you think a neural network, be it natural or artificial,
> accomplishes? What does it do? How is its operation correlated to the
> environment, say, during simple categorization (of audio or images)?

It filters and organises input data. That is, given a mess of inputs,
the NN passes some and blocks others. The output may be "pattern
recognition", or input to another NN. The following description will
show what I understand happens: Given an NN that's been trained to
"recognise the letter A". Input visual data of the letter B. Some of
that data will "pass", that is, activate some connections as the data
flows through the NN (eg, the centre horizontal bar of the B.) Most will
not. The NN will respond (output "Letter A") if enough of the
connections within it are activated and not otherwise. The "reference
state" is the organisation of the connections. Eg, a certain group
inputs are connected to one element, some of the same inputs are
connected to another, and some to a third. If all inputs are on, all
three elements will fire; otherwise only one, or none. NB that my
description assumes a layered architecture. A good example of what I
envision is the mammalian retina, which is a neural network of exquisite
precision. IMO the organisation of the retinal NN is repeated throughout
the brain.

The "correlation to the environment" is a phrase that I don't find very
meaningful. There are organised inputs from the sensors. If the sensors
provide inputs such that the "A-recognition NN" responds, then the
system has "seen" the letter A.

And categorisation of audio or images is _not_ simple. Anyone who has
tried to write a categorisation tree to distinguish between CAT and DOG
should know that. I've doodled some myself, and I've found that at every
level there are assumptions about what the subcategory "means", ie, each
  node hides another tree. Add to that the problem of recognising a CAT
or DOG from variable visual data, and the problem is seen to be
horrendously complex. It's no accident that optical NNs have barely been
ablt to recognise letters, and even those not reliably. IMO, the central
problem of NNs is not their general organisation or properties, it's the
nasty little details of how to actually connect the actual elements.
Small differences in connection topology may have large effects on the
NNs operation; and those effects aren't easily predictable. It's much
easier to refer to "emergent properties" than to design a device that
will display those emergent properties.

Corrections and clarifications welcome.

Relevant Pages

  • Re: Neural netss (was Re: death of the mind.)
    ... >> others refer to is as I understand it the organisation of the NNs in the ... > percept/reference state in the brain. ... The way the entire percept in the cortex ... Note that the face recognition module is not as ...
    (sci.cognitive)
  • Re: Why Software Is Bad and What We Can Do to Fix It
    ... > Forgetting is due to a slow decay of synaptic strength or to some ... Yes, it is obvious that many neurons fire at the same time, but the rest ... > occurring in the brain at any one time, i.e., synchronously. ... same time without reference to the timing of the events, ...
    (comp.object)
  • Re: Origin of the word "Creationism"
    ... reference this fact from 20 different sources at the drop of a hat... ... evolution created the brain and a faculty ... which invented God. ...
    (talk.origins)
  • Re: Neural netss (was Re: death of the mind.)
    ... Once it is agreed that the brain contains a reference state we can ... Note that the face recognition module is not as ... In binocular rivalry the entire percept switches dramatically, ...
    (sci.cognitive)
  • Re: Origin of the word "Creationism"
    ... Produce the 20 reference sources or shut up. ... evolution created the brain and a faculty ... How do you diagnose brain damage, how do you determine that your own brain ...
    (talk.origins)