= Solution
Choose $x_0$, fixed $\bar\alpha>0$, $\sigma,\beta\in(0,1)$, and repeat for every $n\geq0$: compute $g_n=\nabla f(x_n)$, choose $\alpha_n$ by the <backtracking line search> in part (b) with $s_n=-g_n$, and set $x_{n+1}=x_n-\alpha_ng_n$. At a stationary point keep the sequence constant.
Assume $f\in C^1$ has a globally <Lipschitz gradient> and is bounded below on its initial <sublevel set> $\{x:f(x)\leq f(x_0)\}$. Then the <steepest descent method> satisfies $\boxed{\nabla f(x_n)\to0}$. Indeed, the <Armijo condition> and the uniform lower step bound give
$$
f(x_n)-f(x_{n+1})\geq\sigma\alpha_n\|g_n\|_2^2\geq\sigma c\|g_n\|_2^2.
$$
The objective values decrease and stay in the stated sublevel set. Summing the inequality and using its lower bound $f_{\inf}$ yields
$$
\sum_{n=0}^\infty\|g_n\|_2^2\leq\frac{f(x_0)-f_{\inf}}{\sigma c}<\infty,
$$
which proves the limit. Every cluster point is a <stationary point> by continuity of the <gradient>. If the initial sublevel set is compact, cluster points exist. This global convergence result guarantees stationarity from arbitrary initial points under these assumptions; it does not, for a general nonconvex objective, guarantee convergence of the whole iterate sequence or a global minimum.
Back to article page
