= Gauss-Newton algorithm
{c}
{tag=Quasi-Newton method}
{title2=$(J_k^TJ_k)s_k=-J_k^TR(x_k)$}
{wiki=Gauss–Newton_algorithm}
= Gauss-Newton method
{c}
{synonym}
For <nonlinear least squares> $f(x)=\tfrac12\|R(x)\|_2^2$, with residual <Jacobian matrix> $J=DR$, the method approximates the Hessian by $J^TJ$. The exact Hessian is $J^TJ+\sum_jR_j\nabla^2R_j$. Full column rank of $J$ makes the approximation positive definite. At a zero-residual solution with full-rank Jacobian, $J(x_k)^TJ(x_k)\to\nabla^2f(x_*)$, so any convergent sequence of full Gauss-Newton steps satisfies the <Dennis-Moré condition> and converges superlinearly. A nonzero residual at the solution leaves a generally nonvanishing discarded Hessian term; the same conclusion does not follow automatically.
Back to article page
