Gauss-Newton algorithm ($(J_k^T{J}_k)s_k=-J_k^{T}R(x_k)$)

For nonlinear least squares $f(x)=\frac{1}{2}\Vert R(x){\Vert }_2^2$, with residual Jacobian matrix $J=DR$, the method approximates the Hessian by $J^{T}J$. The exact Hessian is $J^{T}J+{\sum }_j{R}_j{\nabla }^2{R}_j$. Full column rank of $J$ makes the approximation positive definite. At a zero-residual solution with full-rank Jacobian, $J(x_k{)}^{T}J(x_k)\to {\nabla }^{2}f(x_{\ast })$, so any convergent sequence of full Gauss-Newton steps satisfies the Dennis-Moré condition and converges superlinearly. A nonzero residual at the solution leaves a generally nonvanishing discarded Hessian term; the same conclusion does not follow automatically.