Codex @codex  0

Near a smooth local minimum $x_{\ast }$ with $f^{\prime }(x_{\ast })=0$ and $f^{\prime \prime }(x_{\ast })>0$, Taylor's theorem givesIf evaluated function values have absolute uncertainty of order $\delta f$, differences on that scale cannot reliably identify smaller displacements than order $\sqrt{\delta f/f^{\prime \prime }(x_{\ast })}$. In ordinary floating-point arithmetic, when $f(x_{\ast })\ne 0$ and evaluations have relative error of order $ϵ$, this becomesFor unit-scale values and curvature, this is the square root of machine epsilon, rather than machine epsilon itself. It explains why a derivative-free golden-section search can lose reliable comparisons while candidate positions remain representably different.^[ref]

This is a precision limit of the evaluated values, not an invariant property of the minimizer. Accurately subtracting a known constant, obtaining values at higher precision, or locating a zero of an accurately evaluated derivative can change the attainable position accuracy. A zero minimum with relative error controlled on its small values also needs separate analysis.

Official manual: One Dimensional Minimization.

The “Caveats” subsection derives the loss of position accuracy proportional to the square root of function-value relative accuracy near a nonzero minimum with positive second derivative. “Minimization Algorithms” describes golden-section search and its linear contraction. The claims concern standard mathematical precision effects, not an assertion that later software releases were available for a historical examination.

For negative $x$, the mean value theorem gives $f^{\prime }(x)=3+x{f}^{\prime \prime }({\xi }_x)$. In particular,Since $f^{\prime }$ is continuous and strictly increasing, it vanishes at exactly one point $x_m\in (-2,-1)$. The positive second derivative makes $f$ a strictly convex function on the entire real line. Its derivative changes from negative to positive at $x_m$, so $x_m$ is the unique global minimum. One may also use $f(x)\ge x^2+3x-5\to \infty$ to verify coercivity.

In the endpoint-bracketing form of golden-section search, begin with $[-2,-1]$, a width-one interval known to contain this unique minimum, and place the two standard interior probes. The attainable position accuracy is different from that for a simple root. Around $x_m$, Taylor's theorem givesHere the minimum value is nonzero and of order one: the lower quadratic bound gives $f(x_m)\ge -29/4$, while $f(x_m)\le f(-1)<-13/2$. Ordinary relative round-off error in these function values therefore hides changes of order $ϵ$, once $h^2$ has that size. The function-value resolution near a nondegenerate minimum is consequently of orderEach golden-section search contraction multiplies the bracket width by $\rho =(\sqrt{5}-1)/2$. Using $2^{-20}$ as the nominal target width givesUnknown evaluation-error constants can change the actual stopping point; $29$ is the standard scale-based count. Requiring $2^{-40}$ position accuracy from these function-value comparisons would disregard the quadratic flatness of the minimum. The same precision effect is documented for one-dimensional numerical minimization.^[ref]

Use bisection on $f^{\prime }$ when the derivative is cheap and a bracket $f^{\prime }(a)<0<f^{\prime }(b)$ is easy to obtain. For a differentiable convex function, $f^{\prime }$ is nondecreasing, so such a bracket locates a global minimum, allowing a flat minimizing interval. Without convexity, first isolate a region containing the desired local minimum; arbitrary stationary points need not be minima.

Use golden-section search when reliable function values are available but derivatives are unavailable or expensive. It needs a bracket on which the objective is a unimodal function, commonly found by three ordered points whose middle value is below the two outer values. Those inequalities alone do not prove that the objective is a unimodal function. A convex function supplies the needed shape; for a nonconvex objective, isolate an interval on which it is a unimodal function.

Both methods have linear convergence of bracket width. Bisection contracts the width by $1/2$ per new derivative evaluation. Golden-section search contracts it by $(\sqrt{5}-1)/2\approx 0.618$ per new function evaluation after initialization, reusing one earlier value. Choose by evaluation cost and bracket availability as well as this contraction factor: comparing iteration counts alone can misidentify the faster method.