= Solution
Put $J(x)=DR(x)$, with one residual gradient per row. The <chain rule> gives
$$
\boxed{\nabla f(x)=\sum_jR^{(j)}(x)\nabla R^{(j)}(x)=J(x)^TR(x).}
$$
Differentiating each product gives
$$
\boxed{\nabla^2f(x)=J(x)^TJ(x)+\sum_jR^{(j)}(x)\nabla^2R^{(j)}(x).}
$$
The approximation $B_n=J(x_n)^TJ(x_n)$ defines the <Gauss-Newton method>. It is symmetric, and full column rank gives $v^TB_nv=\|J(x_n)v\|_2^2>0$ for $v\ne0$, so the step is well-defined.
If $x_n\to x_*$, $R(x_*)=0$ and $J(x_*)$ has full column rank, then $\nabla f(x_*)=0$ and $H_*=\nabla^2f(x_*)=J(x_*)^TJ(x_*)$ is positive definite. Continuity of $J$ gives $B_n\to H_*$. Therefore
$$
\frac{\|(B_n-H_*)s_n\|}{\|s_n\|}\leq\|B_n-H_*\|\to0.
$$
The <Dennis-Moré condition> applies, proving $\boxed{x_n\to x_*\text{ q-superlinearly}}$, or finite termination. The zero-residual hypothesis is what makes the discarded second-derivative term vanish at the limit.
Back to article page
