Local quadratic convergence of Newton iteration ($|e_{r+1}|\le \frac{M}{2m}|e_r{|}^2$)

Let $F$ be twice continuously differentiable on an interval around a zero of a function $t_{\ast }$, with $|F^{\prime }(t)|\ge m>0$ and $|F^{\prime \prime }(t)|\le M$ there. Choose a closed interval $[t_{\ast }-\rho ,t_{\ast }+\rho ]$ inside that interval with $M\rho /(2m)<1$. Starting inside it makes the Newton root-finding iteration well defined, keeps it inside, and gives the displayed quadratic convergence bound for $e_r=t_r-t_{\ast }$. The nonzero derivative and sufficiently close start are essential hypotheses. The scalar estimate follows from the Taylor remainder argument in MIT's root-finding notes.^[ref]

For a short proof outline, apply Taylor's theorem between $t_r$ and $t_{\ast }$; subtract the defining tangent line equation. The linear error cancels, leaving $e_{r+1}=F^{\prime \prime }({\xi }_r)e_r^2/(2F^{\prime }(t_r))$. The displayed bound shrinks the error and preserves the chosen interval. If $F^{\prime \prime }(t_{\ast })\ne 0$, the error ratio tends to $|F^{\prime \prime }(t_{\ast })/(2F^{\prime }(t_{\ast }))|$; if it vanishes, convergence can be faster.

For the strictly convex polynomial $f(x)=x^2/2+x^4/4$, applying Newton root-finding iteration to $f^{\prime }(x)=x+x^3$ givesThe unique minimum is at zero. For any sufficiently small nonzero initial point, the errors satisfy $|x_{k+1}|/|x_k{|}^3\to 2$, so the order is cubic, not merely quadratic. This provides a simple counterexample to treating the usual quadratic convergence theorem as an upper bound on possible orders. In general, the quadratic coefficient for Newton optimization near a nondegenerate stationary point is $f^{(3)}(x_{\ast })/(2f^{\prime \prime }(x_{\ast }))$; when it vanishes, a higher order is possible.