Re: Nonlinear Least-Squares curve-fitting


Scott Seidman wrote:
cafeinst@xxxxxxx wrote in news:1158947866.574595.138520
@h48g2000cwc.googlegroups.com:

This technique was suggested to me in 1992 when I was having trouble
getting my computer program to give a nonlinear least-squares fit to a
certain complicated function. And it worked amazingly. I'd like to know
if anyone has heard of this?

Craig



Never heard of this, but I would think that whether it would work nicely or
not would depend upon how well-behaved the function and the data are--
sometimes it might work nicely, and sometimes not-so-nicely.

I have also tried it on other types of functions, but there didn't seem
to be any gain in doing it this way. The intuition for this type of
technique is that if you start with a small number of data points, the
least squares function is not as complicated as if you had started with
a large number of data points; therefore, if the least squares function
involves a small number of data points, there won't be as many local
minima to avoid as if the least squares function involved a large
number of data points.

It's using *gradualness* to solve the problem. It's actually quite
natural and probably explains why it is not good to teach 1st graders
how to read by giving them "Huck Finn". It's better to first teach them
"Hellicopters and Gingerbread" or "A Duck is a Duck" and let them read
"Huck Finn" in high school. This concept can be used for Machine
Learning too, I would think.

Craig


Regardless of the method you use to generate your initial guess, best
practice is to start from a variety of locations in n-space, and see which
initial guess gives you the smallest least squares error. You usually
don't want to depend on just one initial guess to give you the "right"
answer, even if you have a nifty algorithm like the one you described.
Choosing that handful of starting points can be something of an art, too.

There are some algorithms that actually provide the estimator with enough
"energy" to jump out of local minima. You see this type of thing often
with neural nets, where there can be a hundreds of (essentially
meaningless) weights to estimate, and there are local minima all over the
place.

--
Scott
Reverse name to reply

.