Past exam of the continuous optimisation course of the University of Oxford / 2025 / 1 / c

If $g=\nabla f(x)\ne 0$, the positive-definite symmetric matrix $B$ is an invertible matrix and $s=-B^{-1}g\ne 0$. Multiplying $Bs=-g$ by $s^T$ givesHence $s$ is a descent direction. If $g=0$, then $s=0$ and the algorithm has already reached a stationary point; the strict descent assertion applies before termination.

Set $p=x^{k+1}-x^k={\alpha }^k{s}^k$ and $y=\nabla f(x^{k+1})-\nabla f(x^k)$. The Quasi-Newton secant equation is $B^{k+1}p=y$. If $B^{k+1}$ is a positive-definite symmetric matrix and $p\ne 0$, it follows thatThe weak curvature part of the Wolfe conditions ensures precisely this necessary condition:The last product is positive because ${\alpha }^k>0$, ${\beta }_2-1<0$, and $s^k$ is a descent direction. This explains why a Quasi-Newton method often uses the Wolfe conditions rather than only the Armijo condition.

Interpret boundedness above uniformly: there is an $M<\infty$ such that $B^k⪯MI$ for every $k$. Since $B^k$ is a positive-definite symmetric matrix, the spectral theorem implies $(B^k{)}^{-1}⪰M^{-1}I$. ThereforeCombining the Armijo condition with ${\alpha }^k\ge C>0$ yieldsIf $f\ge L$, a telescoping sum givesThe nonnegative series converges, so its terms tend to zero:This proves first-order global convergence in the sense of stationarity from uniformly bounded quasi-Newton matrices. It does not imply that every nonconvex objective function reaches a global minimum, or that the iterates themselves converge. Every finite limit point is a stationary point, by continuity of the gradient.