Newton step ($H(p)s=-\nabla f(p)$)

A Newton step for Newton's method in optimization is the displacement $s$ solving the displayed linear system, where $H$ is the Hessian matrix and $\nabla f$ the gradient. The next iterate is $p+s$. This makes the first order Taylor expansion of the gradient at the new point vanish. An invertible Hessian matrix determines a unique step; it need not give a descent direction when the Hessian matrix is indefinite.

For a real symmetric invertible matrix $H$, the full undamped Newton step for the displayed quadratic function sends every starting point to the unique stationary point:This follows because the gradient is affine and its first-order expansion is exact. If $H$ is a positive-definite matrix the result is the unique global minimum; a negative-definite matrix gives a global maximum, and an indefinite matrix $H$ gives a saddle point. Thus exact arrival does not by itself show that a minimization method has minimized anything. A line search, damping, or a positive-definite matrix replacing the Hessian matrix changes this exact one-step assertion.