Re: HELP: Non-linear regression problem


Scott Seidman wrote:
"Reef Fish" <Large_Nassau_Gr0uper@xxxxxxxxx> wrote in
news:1158174430.755815.326690@xxxxxxxxxxxxxxxxxxxxxxxxxxxx:


What's so impolite about the straightforward comment you cited?

There is nothing impolite in my citation, as I removed the offensive
material, leaving the interested to turn toward search engines to recover
it.

In that case, it made your comment at best a waste of bandwidth, and
in reality a gratuitous comment without substantiation.

Cut the rhetoric and get on with the statitsical SUBSTANCE.


You have NOT indicated what GAIN you have in your Jacobian computation
over the standard regression methods and outputs.


You're blathering, Bob.

Is that what you call POLITE speech, right after your hypocritical
lecture?

I am not talking about applying a gain to the
Jacobian to calculate the next estimate (assuming that's what you're
getting at, as JACOBIAN CALCULATIONS DON'T HAVE A GAIN).

I meant your END results after your jacobian computation!! The end
result
of your CVs.

Many estimation
algorithms, of course, do this, and I leave that to the algorithms-- though
often you can provide some sort of multiplier for the first value for the
algorithm to use, which sometimes helps jump over false minima, or can
speed up convergence. I merely said that in many cases, you've already
computed the Jacobian for the estimation algorithm, so you don't need to do
it special for CV calculation.

They WHY were you suggesting the computation to the OP, and you argued
that in ordinary Linear Regression Models that your CVs are equally
useful?


If you can't decipher my feeble attempts at plain text representation of
the equations, the Jacobian is plainly defined at
http://mathworld.wolfram.com/Jacobian.html. Look Ma, no gain.

I already knew there is no gain. That was my point about YOUR post.
I gave you ALL the computational and necesseary theoretical details
as well as the references about multicolllinarity and the paper of
Beaton, Barone, and Rubin ... ALL pointing to the fact that there is
NOTHING to your suggestion (but Quackery) in your suggestion and
further obfuscation of the use or advantage of calculating the CVs
as ADDITION information over the conventional SEs and other
analytic measures of both the NUMERICAL and STATISTICAL
stability of the Linear Models Regression situation.

In the Nonlinear Regression context, it's inverse of the Fisher
Information matrix, which is a kind of Jacobian to yield information
analoguous to the multicollinearity condition, except the
interdependence is no long of the LINEAR type.

All of this is the STANDARD material in the graduate course I've
taught many times.

Your next paragraph is just more non-technical rhetoric and
obfuscation of the Richard Ulrich type.

All the CV is is a standard error normalized by the estimate. If you
provide the SE and the estimate, you've essentially provided the CV, and
thus provided some information on precision. If you've provided the
estimate and a p-value on that estimate, you're not providing as much
information, and the info you've left out robs your reader/client/whatever
of a tool that could help provide an assessment of the quality of your
estimate. That's all I've said, or at least what I've been trying to say.
There it is in one paragraph. If you would like to criticize that, a
polite individual would address those points of disagreement, and state
what's wrong with them.

I had already pointed out the LAST time, and in the above THIS time,
what's wrong with your CV approach.


I've run the examples I cited before in Matlab, fitting
5*exp(-t/2.5)+1+noise to the single decaying exponential with an offset

ALL your Excel computation is the classic "Garbage IN, Garbage OUT"..

It's the mathematical OBFUSCATION the analogue of your verbal
obfuscation.

First of all, your example does NOT even have an unknown parameter to
be
FITTED! Where is the PARAMETER? Are you just another Afonso?

If your model is a single parameter exponential decaying model, then it
is
a well-known case of a nonlinear model that CAN be linearized. See
all the LInear Models books in statistics that uses THAT example as the
simplest one to illustrate the idea of a non-intrinsic nonlinearity.

Your NUMERICAL GARBAGE is kept just to preserve that that they
are ALL garbage.
================= Begin Scott Seidman garbage
X(1)=A=5
X(2)=offset=1
X(3)=time constant=2.5
[3 1 1] is my initial guess for X
decayeh is an anonymous function E=y-yhat

lsqnonlin, by default, uses an interior-reflective Newton method, but this
makes little difference. I opted not to provide the Jacobian for the
minimization, but calculated it post-hoc. I didn't express CV's as a
percent. Feel free to multiply by 100 if you want.

decayeh= @(x ,t, noise) (5*exp(-t/2.5)+1+noise)-(x(1)*exp(-t/x(3))...
+ x(2));
decayeh
t= [0:1000]/100;
noise=.1*randn(size(t));
x1=lsqnonlin(decayeh,[3 1 1],[],[],[],t,noise)
dydA=-exp(-t/x1(1));
dydb=ones(size(t));
dydtau=(x1(1)/x1(3)/x1(3).*t.*exp(-t/x1(3)));
jac=[ dydA' dydb' dydtau'];
varcov=inv(jac'*jac)
cv_A=sqrt(varcov(1,1))/x1(1)
cv_b=sqrt(varcov(2,2))/x1(2)
cv_tau=sqrt(varcov(3,3))/x1(3)

decayeh =

@(x ,t, noise) (5*exp(-t/2.5)+1+noise)-(x(1)*exp(-t/x(3)) + x(2))

Optimization terminated: relative function value
changing by less than OPTIONS.TolFun.

x1 =

5.0133 0.9948 2.5099


varcov =

0.0213 0.0040 0.0115
0.0040 0.0069 -0.0092
0.0115 -0.0092 0.0311


cv_A =

0.0291


cv_b =

0.0835


cv_tau =

0.0703

If I increase the noise to var=1
x1 =

5.0390 1.0040 2.5165


varcov =

0.0213 0.0041 0.0114
0.0041 0.0070 -0.0092
0.0114 -0.0092 0.0310


cv_A =

0.0290


cv_b =

0.0831


cv_tau =

0.0700


If I cut off the first 2 seconds of data, and var(noise)=0.1
decayeh =

@(x ,t, noise) (5*exp(-t/2.5)+1+noise)-(x(1)*exp(-t/x(3)) + x(2))

Optimization terminated: relative function value
changing by less than OPTIONS.TolFun.

x1 =

5.0522 1.0089 2.4611


varcov =

1.2541 0.0160 0.9003
0.0160 0.0075 -0.0018
0.9003 -0.0018 0.6756


cv_A =

0.2217


cv_b =

0.0856


cv_tau =

0.3340

Thus, losing precision on the time constant and gain, but we're still
relatively OK for the offset


If I just include the first 4 seconds of data, var(noise)=0.1:
decayeh =

@(x ,t, noise) (5*exp(-t/2.5)+1+noise)-(x(1)*exp(-t/x(3)) + x(2))

Optimization terminated: relative function value
changing by less than OPTIONS.TolFun.

x1 =

4.9850 1.0304 2.4603


varcov =

0.3034 0.3347 -0.2095
0.3347 0.3971 -0.2732
-0.2095 -0.2732 0.2146


cv_A =

0.1105


cv_b =

0.6116


cv_tau =

0.1883

So, we've lost a bit of precision on A and tau, but the offset precision is
shot to ***.

Pretty much behaves just like it should, huh?
============== End of Scott Seidman garbage ========


--
Scott
Reverse name to reply

If you want to demonstrate the VALUE (that is nonzero) in your CV
approach
in a NONLINEAR regression problem that is intrisically nonlinear, take
some
REAL data and fit it to the Multicompartment model of the SUM of two
expoential decay model, with FOUR unknown parameters (I assume you
know how to write that model or can find out from the literature that
is one
of the most common model that presents all kinds of instability
problems
in the ESTIMATED parameters).

If you can take a set of data in the literature that had been analyzed
the
traditional way, so much the better. THEN, try to make you case that
by redoing the same problem YOUR way, and point out what advantage
it has.

Of course you CAN'T. I am VERY familiar with that problem and the
difficult of estimation of the parameters intrisic in THAT particular
model.
That was one of my "consulting projects" at Yale Statistics Department
(when I was a grad student) helping faculty in OTHER departments with
their valid statistical problems -- under the supervision of a faculty
member -- as part of the CURRICULUM in the Applied Statistics
Program -- of spending a FULL HALF YEAR doing nothing but
consulting (under supervision). In my case, the supervisor was
L.J.Savage who knows that problem as well as anyone else in the
world. I've encountered that nonlinear problem and many other
nonlinear problems in my applied work both in teaching and research.

Scott, what you're doing here is just KEEP DIGGING a deeper
and deeper Hole after you found yourself in one.

Your excel example is the kind that is is not even beyond a junior
undergrad level in statistics, to use to bluff you way out -- that
MIGHT have worked (in fact, I suspect it would have worked in this
group -- had there not been someone who could see throught ALL
your Quackery within a nanosecond.

The ABSENCE of some "watch dog" in this group from 1994 to
2005 is the reason Richard Ulrich had perpetrated so many
Quackery and malpractice and NOT been called to make himself
accountable until I came into this group in 2005.

Scott, I've pointed out your errors BEFORE, on much simpler models
and standard material on regression. You ABSENCE of knowledge
about NONlinear regression and your Quackery of advicing some
innocent victim about your MISUSE of CV is NOT going to get away
no matter HOW your throw your ink (as an octopus) to cover your
misguide track.

First Law of Holes: "If you find yourself in a Hole, STOP DIGGING."


You're welcome to come back with an example, on the double exponential
decay model, with real DATA, to demonstrate that your CV idea is NOT
quackery.

Failing to do that, your FREE tuition will be withdrawn -- because you
have already demonstrated beyond a reasonable doubt, that your
knowledge about nonlinear regression (or even Linear Regression) is
nothing beyond your ability to stuff some numbers into Excel or some
other statistical package or program, and exhibit that YOU are one of
those who had mastered the technique of "GARBAGE IN, GARBAGE OUT".

-- Reef Fish Bob.

.

Loading