Re: ? non oredr-2 obj function


In article <47f04fe9$0$17358$4c368faf@xxxxxxxxxxxxxx>,
"Cheng Cosine" <acosine@xxxxxxxxxx> writes:
Hi:

2nd order objective functions were used quite often in optimization

due to the fact that one can then solve linear equations to determine

critical point(s) satisfying the necessary condition for optimization.
?????? do you mean a C^2 function or a quadratic approximation of
a C^2 or even less smooth function?

or simply this: solve grad f(x)=0 by a generalized Newton's method?
or simply: is it better to use a second order (or superlinerly
convergent) method instead of a linearly converging one which uses
only gradient information?




However, in terms of sensitive of objective function, isn't it true

that higher order objective functions are more sensitive? Besides,

what do you mean by sensitivity of a function? I guess you mean the
sensitivity of the location of a minimizer of C^2 function is
more sensitive to perturbations of the function value than
the position of a strict local minimizer of a piecewise linear
function ???

but a a real problem, the function is given to you and you will
not be able to change its properties!



when one encounters nonlinear constraints, it becomes meaningless

to keep that 2nd order objective function, since even after
differentitation,
this is not true. everything translates easily to the constraint
case, at least in theory, and interior point and sqp methods demonstrate
that also practically this works

one still has those nonlinear equaions from constraints.

Thus, my question is: are there already some evidences in theory or

from experience that non 2nd order objective functions serve better

for some kinds of optim problems?


yes clearly, for all such problems which meet the sufficient conditions for
their application
BTW, if one is sure that the objective function value is never zero, will

square-root serves as good as a quadratic func since they are symmetric

to y = x, so convergence rates (or gradient) are the same?

Thanks,
by Cheng Cosine
Mar/30/2k8 NC



you mean: solve f(x)= min by the gradient method
of
solve sqrt(f(x)) = min by the gradient method
givne that
f(x) >= eps >0 for all x?
well:
compute the spectrum of the Hessian and this will give you the answer:

grad (sqrt(f(x)) = 0.5/sqrt(x) grad f(x)
Hess (sqrt(f(x)) = -0.25/sqrt(x)^3 grad f(x) grad f(x)' + 0.5/sqrt(x)*Hess f(x)

inserting x^* as a local opitmizer you get grad f(x^*)=0
hence the two spectra discern by the factor 0.5/sqrt(f(x^*)) which does
not influence the local convergence rate

but

in Hess (sqrt(f(x)) a positive semidefinite matrix of rank one is
subtracted form the Hessian of f and depending on size this may worsen
the global properties of the problem essentially (for n>=2)

hth
peter

.