Re: Neural netss (was Re: death of the mind.)

From: dan michaels (feedbackdroids_at_yahoo.com)
Date: 09/28/04 

Date: 28 Sep 2004 14:12:53 -0700

Wolf Kirchmeir <wwolfkir@sympatico.ca> wrote in message news:<gAy5d.1973$MD5.21482@news20.bellglobal.com>...

> >
> > If we are to talk about electronic devices, it might help to think of
> > the brain as a PAL (programmable array logic} rather than a Von
> > Neumann computer. Then the signal energy comes in through the sensory
> > neurons, filters through the interneurons, and comes out as a motor
> > act. (Majendie's Law) No magic, just circuitry.
>
> That looks like a Better Idea, all right. Just how complexly
> interconnected can one make the elements of a PLA? Does each element
> have a single input and a single output? If so, it's not IMO complex
> enough to do what biological NNs do. See below.
>

Neither approach is very good. Obvious for the von Neumann approach.
Also, PLA's and PAL's are not all that powerful. Made for implementing
small sets of combinational logic is all. A few inputs and a few
outputs. I seriously doubt the brain is a combinational logic machine.
Misnomer still extant from McCulloch+Pitts original 1943 paper. The
first sentence of the paper starts out badly .....

"... Because of the 'all-or-none' character of nervous activity,
neural events and the relation among them can be treated by means of
propositional logic ..."

Time to move on. Interesting that McCulloch was an M.D., if I'm not
mistaken.
==================

> > Now we have to think how we should abstract groups from this amorphous
> > mass of interneurons. We can talk about motor program generators,
> > initiators, and controllers--not word processing. What circuitry do we
> > need so that a motor program generator can be modified by experience?
>
> Feedback _networks_ (not simple loops.)
>

Good answer. Had "I" made it, King David would be bouncing off the
walls again, about now. Good to see so many people are picking up on
the feedback thingie.
==================

> > What circuitry to throttle the signal energy temporarily as it rushes
> > through one path so that it also has time to run through an
> > alternative path?
>
> A simple loop NN can hold a signal spike indefinitely: Ni -->Nx --> Ny
> --> Nx will hold the input from Ni as long as the NN runs. But we also
> need some Nx -->Ny so that the signal can eventaully go someplace else.
> This means that any neuron can have more than one output. That's the key
> IMO.
>

Yes, and in the real brain, take what you just said and scale it by
about a billion times ... or more.
=================

> NB that this is the only way a NN can store data, and then storage is in
> fact a constant cycle through the same sequence, which is not what
> storage means in ref. to computers. RAM is refreshed, but that's not the
> same process. In actual biological NNs the signal eventually decays.
>

Loops are good, but what about synaptic modifications?
==============

> > How does the signal energy actuate the throttle? How
> > does it disable the throttle?
>
> Use inhibiting signals as well activating ones. Each neuron has one
> input, which requires a certain activation strength before it fires. Use
> a few holding loops to accumulate signals until there are enough to fire
> some downstream neuron Nk. Use inhibiting feedback from Nk to the
> holding loops to switch them off so that a new signal can be held. NB,
> again, that Nk must have at least two outputs. In general any given
> neuron will have a single input and one or more outputs. It's the
> multiple outputs that enable the topology you're looking for. In
> biological NNs, most neurons have two or more outputs.
>

Strange statement ... "... In general any given neuron will have a
single input ...". 5,000-10,000 inputs is more like it. And
1,000-2,000 outputs.

So, given this, how would you revise your description above, regards
your simple loop? ..... Ni -->Nx --> Ny --> Nx

Now, we have a really interesting problem.
=================

> Problem is, AFAIK there's no simple way to describe such networks. Graph
> theory can characterise the topology of networks (by type of
> connectivity, for example), but AFAICT it can't handle the actual
> topology of a NN that's complex enough to do interesting things.
>
> > ray
>

Something, in a mathematical sense, inbetween graph theory and
descriptive equations for kinetics of gases. Complexity theory?
================

> The above ideas are _not_ original with me. I first came across them in
> the early to mid-60s in a book title The Minds of Robots, which
> disappeared from the university library shortly after I returned it, so
> I never found it again. Can't recall the author, but do remember the
> book was published in Bloomington.

Relevant Pages

  • Re: Neural netss (was Re: death of the mind.)
    ... This means that any neuron can have more than one output. ... In actual biological NNs the signal eventually decays. ... Use inhibiting signals as well activating ones. ... neuron will have a single input and one or more outputs. ...
    (sci.cognitive)
  • Re: Neural netss (was Re: death of the mind.)
    ... >> This means that any neuron can have more than one output. ... >> Use inhibiting signals as well activating ones. ... >> neuron will have a single input and one or more outputs. ... >> multiple outputs that enable the topology you're looking for. ...
    (sci.cognitive)
  • Re: consciousness
    ... brain, but the thing we are sensing normally starts "out there". ... the signals is irrelevant to our conscious awareness. ... If you have two electrons in a wire, and one moves, it can cause the one ... But you haven't explained what happens if you take that neuron out of the ...
    (comp.ai.philosophy)
  • Re: Symbolic AI: Why Marvin Minsky and Curt Welch Are Out to Lunch
    ... a pyramidal neuron in the sensory cortex is strongly increased if they ... fire about 10 ms before the target neuron fires. ... significantly correlated signals from two related streams. ...
    (comp.ai.philosophy)
  • Re: Access bus signal names in S-function
    ... It may turn out easier/less messy to call the m-function ... a single input. ... access the signal names if I have a bus of many signals ... I can access the InputSignalNames from the originating ...
    (comp.soft-sys.matlab)