Descent direction ($\nabla f(x{)}^{T}s<0$)

A vector $s$ is a descent direction for a differentiable function $f$ at $x$ if its directional derivative $\nabla f(x{)}^{T}s$ is negative. The first-order expansion $f(x+\alpha s)=f(x)+\alpha \nabla f(x{)}^{T}s+o(\alpha )$ then gives $f(x+\alpha s)<f(x)$ for sufficiently small positive $\alpha$. The negative gradient is a descent direction whenever the gradient is nonzero.

For a nonzero gradient, this unit direction minimizes the directional derivative among Euclidean unit directions. The Cauchy-Schwarz inequality gives $\nabla f(x{)}^{T}p\ge -\Vert \nabla f(x){\Vert }_2$, with equality for the displayed direction. The steepest descent method usually uses the equivalent unnormalized direction $-\nabla f(x)$ and chooses a separate step length. A different choice of norm can change the steepest direction.