= Function-value resolution near a nondegenerate minimum
{title2=$\delta x\asymp\sqrt{\delta f/|f''(x_*)|}$}
Near a smooth <local minimum> $x_*$ with $f'(x_*)=0$ and $f''(x_*)>0$, <Taylor's theorem> gives
$$
f(x_*+h)-f(x_*)=\tfrac12 f''(x_*)h^2+o(h^2).
$$
If evaluated <function> values have absolute uncertainty of order $\delta f$, differences on that scale cannot reliably identify smaller displacements than order $\sqrt{\delta f/f''(x_*)}$. In ordinary <floating-point arithmetic>, when $f(x_*)\ne0$ and evaluations have relative error of order $\epsilon$, this becomes
$$
|h|\sim\sqrt{\epsilon\,\frac{|f(x_*)|}{f''(x_*)}}.
$$
For unit-scale values and curvature, this is the square root of <machine epsilon>, rather than machine epsilon itself. It explains why a <derivative>-free <golden-section search> can lose reliable comparisons while candidate positions remain representably different.<One dimensional minimization in the GNU Scientific Library manual>[Caveats: loss of the quadratic term when added to a nonzero minimum value]{ref}
This is a precision limit of the evaluated values, not an invariant property of the minimizer. Accurately subtracting a known constant, obtaining values at higher precision, or locating a zero of an accurately evaluated <derivative> can change the attainable position accuracy. A zero minimum with relative error controlled on its small values also needs separate analysis.
Back to article page
