Re: time-series smoothing

From: Peter Spellucci (spellucci_at_fb04373.mathematik.tu-darmstadt.de)
Date: 03/18/05 

Date: Fri, 18 Mar 2005 19:34:51 +0000 (UTC)

In article <iEGoNLAGYdOCFwWI@mcdowella.demon.co.uk>,
 "A.G.McDowell" <mcdowella@mcdowella.demon.co.uk> writes:
>In article <4239ACF1.55B96907@ever.com>, Jake <wh@ever.com> writes
>>I have a problem of an apparent cross-disciplinary nature so I hope that
>>justifies the cross-posting.
>>
>(trimmed)
>>
>>2] Would it make sense to use Nelder-Mead (downhill simplex) method for
>>minimizing, even though this problem seems to be a constrained
>>optimization? There are three parameters, all constrained to positive
>>values and one constrained to integer values, so differentiation doesn't
>>seem to be an option. Is there a better derivative-free optimizer for
>>this problem? Is there a differentiation method that would work better,
>>even for a function that is only partially differentiable?
>>
>I note that the Nelder-Mean algorithm is mentioned in Numerical Recipes
>to need restarting in practice and is now known to sometimes fail to
>converge (google on Nelder simplex counter-example yields e.g.
>http://portal.acm.org/citation.cfm?id=589108: the body of the text
>requires a subscription, but the abstract is accessible to all). Torczon
>has provided a variety of convergence proofs for direct search
>algorithms, one of which looks like a variant of the Nelder-Mead
>algorithm: googling on Torczon Simplex yields http://www.cs.wm.edu/~va/r
>esearch/, which includes a paper "On the convergence of the
>multidirectional search algorithm".
>
>Are these methods destined to replace Nelder-Mead? Nice Theoretical
>curiosities? Something in between?
>--
>A.G.McDowell
 
there are several convergent variants of Nelder Mead known now:

Zbl 0962.65048 Kelley, C.T.
Detection and remediation of stagnation in the Nelder-Mead algorithm
using a sufficient decrease condition. (English)
SIAM J. Optim. 10, No.1, 43-55 (1999).
(no guarantee of convergence though)

20. Zbl 1030.90122 Tseng, Paul
Fortified-descent simplicial search method: A general approach. (English)
SIAM J. Optim. 10, No.1, 269-288 (1999). MSC 2000:
(a mix of Torczon like steps with the Nelder Mead idea, in practice more
like a grid search)

Zbl pre01812445 Price, C.J.; Coope, I.D.; Byatt, D.
A convergent variant of the Nelder--Mead algorithm. (English)
J. Optimization Theory Appl. 113, No.1, 5-19 (2002)
(with proof of convergence, only mnor modifications to the original nelder Mead)

hth
peter

Relevant Pages

  • Re: time-series smoothing
    ... >I note that the Nelder-Mean algorithm is mentioned in Numerical Recipes ... >has provided a variety of convergence proofs for direct search ... one of which looks like a variant of the Nelder-Mead ...
    (sci.stat.edu)
  • Re: time-series smoothing
    ... I'm wondering about fitting before optimization to reduce the effect ... >>I note that the Nelder-Mean algorithm is mentioned in Numerical Recipes ... >>has provided a variety of convergence proofs for direct search ... one of which looks like a variant of the Nelder-Mead ...
    (sci.math.num-analysis)
  • Re: time-series smoothing
    ... Is there a differentiation method that would work better, ... I note that the Nelder-Mean algorithm is mentioned in Numerical Recipes ... Are these methods destined to replace Nelder-Mead? ...
    (sci.math.num-analysis)
  • Re: time-series smoothing
    ... Is there a differentiation method that would work better, ... I note that the Nelder-Mean algorithm is mentioned in Numerical Recipes ... Are these methods destined to replace Nelder-Mead? ...
    (sci.stat.edu)
  • Re: request for algorithm
    ... >that can be solved by means of N-dimensional iterative algorithms, ... but when using only previous points convergence gives me ... >algorithm ensures convergence. ...
    (sci.math.num-analysis)