Past exam of the continuous optimisation course of the University of Oxford / 2025 / 2 / a / iii

The Cauchy point provides both an inexpensive fallback step and a standard against which a better step can be judged. Requiring a fixed Fraction of Cauchy decrease givesConsequently an exact global minimum of each trust-region subproblem is unnecessary. A step from any inner solver is adequate if it meets this bound.

The convergence argument combines this decrease with model accuracy. For example, suppose the gradient is uniformly Lipschitz continuous on a neighborhood of the visited level set, the model Hessian matrices are uniformly bounded, and the usual acceptance and radius safeguards are used. If $\Vert \nabla f(x^k)\Vert \ge ϵ>0$, the Taylor expansion gives a model error bounded by $K\Vert s_k{\Vert }^2$. For sufficiently small ${\Delta }_k$, division by the Cauchy decrease bound makes $|1-{\rho }_k|$ small. Such trials are successful and cannot keep shrinking the radius. With the gradient bounded away from zero, the radius therefore has a positive lower bound, and each accepted step decreases $f$ by a fixed positive amount. There must be infinitely many accepted steps, since repeated rejection would force the radius below the small-radius acceptance threshold. This contradicts boundedness below of $f$ and proves ${\mathrm{liminf}}_k\Vert \nabla f(x^k)\Vert =0$.

Stronger standard conclusions can be obtained with further regularity and safeguards. Merely being twice continuously differentiable and bounded below does not supply uniform Hessian bounds on an unbounded level set; those uniform assumptions must be stated when using this argument.