Re: Intelligence and Statistics

From: Glen M. Sizemore (gmsizemore2_at_yahoo.com)
Date: 09/24/04 

Date: Fri, 24 Sep 2004 17:32:01 -0400

Do you agree that it is reasonable to characterize feature
extraction/discrimination
circuitry as accomplishing (at least in part) an unsupervised statistical
analysis of incoming
sensory data? Do you agree that it is reasonable to characterize the
mechanisms responsible for
conditioning as (again, at least in part) accomplishing a sort of regression
analysis to discover
the relationship between inputs and conditioning signals treated as
dependent variables?

GS: I know you weren't asking me, but I'll answer anyway. No. Just as I do
not think it is reasonable to assert that gravitationally interacting bodies
"implement differential equations."

"Bill Modlin" <modlin1@metrocast.net> wrote in message
news:nMWdnY4WCPabVszcRVn-ug@metrocastcablevision.com...
>
> "David Longley" <David@longley.demon.co.uk> wrote in message
> news:4xPVz4BrYZUBFwKp@longley.demon.co.uk...
> > In article <xsSdnQt0wNBjHMzcRVn-ig@metrocastcablevision.com>,
> > Bill Modlin <modlin1@metrocast.net> writes
>
> > >David Longley, in a recent response to one of my postings:
> > >
> > > "Then why not drop all pretence that you are talking about
> > > "intelligence" and face the fact that you are doing
> > > statistics."
> > >
> > >I've been saying all along that an ongoing process best
> > >understood as a form of statistical inference at the cellular
> > >level is an important element of intelligence, so for me it is
> > >consistent to agree that I'm "doing statistics" while also
> > >claiming to talk about intelligence.
> > >
> > >David's remarks remind me that I have a dusty web page on
> > >which I stored a couple of my older postings to that effect:
> > >
> > > http://www.metrocast.net/~modlin1/
> > >
> > >The links from there to other sites are obsolete, but at least
> > >you can see what I wrote years ago about "the statistical
> > >basis of intelligence" if you are interested.
> > >
> > >David continues:
> > >
> > > "That's the field which looks at the development and
> > > refinement of algorithms which compute the relations between
> > > input (independent) and output (dependent) variables."
> > >
> > >Half true. The sort of statistical algorithms I'm talking
> > >about involve a distinction which David seems unable to grasp.
> > >
> > >David's notion of statistics are of the "supervised" variety:
> > >the dependent variables are given, and the goal is to discover
> > >a specific relationship, to find out how to generate or
> > >predict the given outputs as a function of the corresponding
> > >inputs.
> > >
> > >The algorithms I'm most interested in are "unsupervised".
> > >They are not given a particular dependent variable to predict.
> > >They are given only a collection of independent variables, and
> > >their goal is to generate new previously unspecified dependent
> > >variables as functions of redundant relationships detected
> > >among the independent variables.
> > >
> > >David's statistics implement conditioning. Mine implement
> > >discovery.
> > >
> > >Learning generally requires both sorts, with the unsupervised
> > >processing extracting features to be used as input by the
> > >later supervised stages.
> > >
> > >As a gloss, organisms implemented with fixed (designed or
> > >evolved) feature detection stages followed by fixed output
> > >functions are "rigid", "brittle", "mechanical". Such
> > >an implementation can produce arbitrarily elaborate behavior
> > >appropriate for the particular situations for which it is
> > >designed, but fails to adapt to alterations in circumstance
> > >outside its design range.
> > >
> > >Adding "supervised learning" mechanisms allows conditioning to
> > >introduce new behavior patterns.
> > >
> > >This sort of adaptation is limited to small incremental
> > >changes, since it is dependent on detectable statistical
> > >association between currently discriminated aspects of the
> > >environment (currently detectable "features") and the training
> > >signals. Elaborate behavioral changes are possible, but
> > >only in small steps such that each necessary discrimination is
> > >conditioned at a point in the process where it is accessible
> > >in the surface statistics of previously learned
> > >discriminations.
> > >
> > >Adding "unsupervised learning" mechanisms allows the organism
> > >to adapt more rapidly to changing circumstances, since new
> > >discriminations are continually being developed as functions
> > >of those currently available, independent of the availability
> > >of reward or punishment.
> > >
> > >The sorts of behavioral patterns which we characterize as
> > >"intelligent" arise as a result of the ongoing internal
> > >activity of unsupervised learning, a process of statistical
> > >inference.
> > >
> > >Bill Modlin
> > >
> >
> > As I've said many times before, "supervised" == regression and
> > discriminant analysis, and "unsupervised" == cluster and factor
> > analysis.
> >
> > I'm really not sure that psychologizing or neurologizing such
> > algorithms does much more than constitute those with a
> > relatively poor grounding in behavioural science (and a related
> > fondness of metaphysics) to delude themselves and others that
> > they're doing something else. Substituting non-linear functions
> > (in the supervised class - e.g. logistic regression) for
> > classic linear functions (like Mean Least Squared Error)
> > doesn't fundamentally change this. It just changes how the fits
> > are made (which can be useful as a model of what biologically
> > happens - but see what I have drawn attention to on this). The
> > fact that these functions are sometimes referred to as fractal
> > or strange tends to just feed the metaphysician in folk
> > (although I'm all in favour of sophisticated analysis of phase
> > spaces where it produces useful results - sadly it rarely has).
> > Furthermore, re-badging cluster analysis as "associative
> > memory" is, I suggest just another example of this modern
> > "sexing up" abuse.
> >
> > I think we should call regression analysis "regression
> > analysis" and cluster analysis "cluster analysis. When I've
> > covered this in "Fragments" I tried to point out how these
> > technologies can be used to highlight something more
> > interesting than our disposition to euphemize (or as Wolf might
> > say, "synonymize" ;-). What I reckon we need to work
> > on is a better understanding of behaviour using simple
> > biological systems to get better control over our variables,
> > and drawing on the above algorithms as indispensable elements
> > of the extensional stance in support of modelling and
> > simulation, and where possible, to a lesser extent, behaviour
> > management. But that's more about us behaving "intelligently"
> > as I've said before, and I reckon there's much to be
> > said, as always, in favour of our getting our priorities right.
> >
> > --
> > David Longley
>
> Your characterization of "unsupervised" as only cluster and factor
analysis misses some important
> extensions that I'll address later, but at least you do cede the existence
of such methods. Given
> that, there are some points that you seem to slide by without explicit
acknowledgement:
>
> You speak of our using statistical analysis to model and simulate
behavior, but I was talking about
> biological systems which themselves do such analyses, as part of their
functional role in an
> organism. Do you agree that it is reasonable to characterize feature
extraction/discrimination
> circuitry as accomplishing (at least in part) an unsupervised statistical
analysis of incoming
> sensory data? Do you agree that it is reasonable to characterize the
mechanisms responsible for
> conditioning as (again, at least in part) accomplishing a sort of
regression analysis to discover
> the relationship between inputs and conditioning signals treated as
dependent variables?
>
> "Unsupervised" mechanisms function independently of overt manifestation of
their results in
> externally accessible behavior. The evolutionary point of such mechanisms
is that they sometimes
> aid in the production of useful behavior. But they accomplish their part
of the overall function
> before the visible behavior occurs, and independent of whether the
behavior occurs at all. Which
> suggests that behavioral studies may not be sufficient to fully
characterize the operation of these
> precursor functions. Would you comment on this?
>
> In this post you repeat your call for us to work on better understanding
of behavior in simple
> biological systems, but it seems likely that those simple systems are
missing those functional
> elements which are responsible for the unsupervised preprocessing that I
consider central to
> "intelligence", however loosely that word may be defined.
>
> While all knowledge gained is potentially of value, can you explain more
clearly why, when seeking
> to understand the mechanisms of intelligence, I should specifically focus
on organisms lacking such
> mechanisms?
>
> Bill Modlin
>
>
>
>