= Solution
A <descent direction> $s$ satisfies $g^Ts<0$, where $g=\nabla f(x)$; the first-order <Taylor expansion> then gives $f(x+\alpha s)<f(x)$ for sufficiently small $\alpha>0$. To compare directions one must fix their scale. Among Euclidean unit vectors, the <Cauchy-Schwarz inequality> gives
$$
g^Td\geq-\|g\|_2,\qquad\boxed{d=-g/\|g\|_2}\quad(g\ne0).
$$
This is the unit <steepest descent direction>. The conventional unnormalized direction used in the algorithm is the same ray, $\boxed{s=-g}$. Equivalently it is the unique minimizer of $g^Ts+\tfrac12\|s\|_2^2=\tfrac12\|s+g\|_2^2-\tfrac12\|g\|_2^2$. If $g=0$ there is no strict first-order <descent direction>, and the conventional update is zero.
Back to article page
