Re: time-series smoothing

We are having a similar problem with noise and we have been using
Nelder-Meade. The surface is fairly broad/flat and the data used in the
optimization is noisey (~20dB SNR) and a fairly small number of variables
(<30). I'm wondering about fitting before optimization to reduce the effect
of the noise.

In article <d1fagr$225$1@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>,
spellucci@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx (Peter Spellucci) wrote:
>
>In article <iEGoNLAGYdOCFwWI@xxxxxxxxxxxxxxxxxxxxx>,
> "A.G.McDowell" <mcdowella@xxxxxxxxxxxxxxxxxxxxx> writes:
> >In article <4239ACF1.55B96907@xxxxxxxx>, Jake <wh@xxxxxxxx> writes
> >>I have a problem of an apparent cross-disciplinary nature so I hope that
> >>justifies the cross-posting.
> >>
> >(trimmed)
> >>
> >>2] Would it make sense to use Nelder-Mead (downhill simplex) method for
> >>minimizing, even though this problem seems to be a constrained
> >>optimization? There are three parameters, all constrained to positive
> >>values and one constrained to integer values, so differentiation doesn't
> >>seem to be an option. Is there a better derivative-free optimizer for
> >>this problem? Is there a differentiation method that would work better,
> >>even for a function that is only partially differentiable?
> >>
> >I note that the Nelder-Mean algorithm is mentioned in Numerical Recipes
> >to need restarting in practice and is now known to sometimes fail to
> >converge (google on Nelder simplex counter-example yields e.g.
> >http://portal.acm.org/citation.cfm?id=589108: the body of the text
> >requires a subscription, but the abstract is accessible to all). Torczon
> >has provided a variety of convergence proofs for direct search
> >algorithms, one of which looks like a variant of the Nelder-Mead
> >algorithm: googling on Torczon Simplex yields http://www.cs.wm.edu/~va/r
> >esearch/, which includes a paper "On the convergence of the
> >multidirectional search algorithm".
> >
> >Are these methods destined to replace Nelder-Mead? Nice Theoretical
> >curiosities? Something in between?
> >--
> >A.G.McDowell
>
>there are several convergent variants of Nelder Mead known now:
>
>Zbl 0962.65048 Kelley, C.T.
>Detection and remediation of stagnation in the Nelder-Mead algorithm
>using a sufficient decrease condition. (English)
>SIAM J. Optim. 10, No.1, 43-55 (1999).
>(no guarantee of convergence though)
>
>
>20. Zbl 1030.90122 Tseng, Paul
>Fortified-descent simplicial search method: A general approach. (English)
>SIAM J. Optim. 10, No.1, 269-288 (1999). MSC 2000:
>(a mix of Torczon like steps with the Nelder Mead idea, in practice more
>like a grid search)
>
>
>
>Zbl pre01812445 Price, C.J.; Coope, I.D.; Byatt, D.
>A convergent variant of the Nelder--Mead algorithm. (English)
>J. Optimization Theory Appl. 113, No.1, 5-19 (2002)
>(with proof of convergence, only mnor modifications to the original nelder
> Mead)
>
>hth
>peter
.