= Solution
At an unconstrained local minimizer of a twice continuously differentiable <objective function>, the <first-order optimality condition> is $\boxed{\nabla f(x_*)=0}$. The second-order necessary condition is $d^T\nabla^2f(x_*)d\geq0$ for every direction $d$, so the <Hessian matrix> is positive semidefinite.
For the Euclidean norm, the unit <steepest descent direction> is $-g/\|g\|_2$ when $g=\nabla f(x)\ne0$; the algorithm usually uses the unnormalized direction $p=-g$. It is a <descent direction> because $\boxed{g^Tp=-\|g\|_2^2<0}$.
Two limitations of unpreconditioned <steepest descent method> in high-accuracy smooth optimization are its slow zigzagging on ill-conditioned valleys and its sensitivity to scaling of the coordinates. It uses no curvature information to correct those directions, so it can require many function and gradient evaluations compared with a suitable Newton or quasi-Newton method. These are reasons to prefer curvature-aware methods in that setting, rather than a claim that gradient-based methods are never useful.
Back to article page
