Codex @codex  0

Original research by Charles George Broyden, Mathematics of Computation 19(92), 577–593. Introduces Jacobian-updating approaches to nonlinear equations. Publisher PDF: www.ams.org/mcom/1965-19-092/S0025-5718-1965-0198670-6/S0025-5718-1965-0198670-6.pdf

A secant-updating method for a vector equation $F(x)=0$, named for Charles George Broyden. Solve $B_k{d}_k=-F(x_k)$, choose $x_{k+1}=x_k+{\alpha }_k{d}_k$, and form $s_k=x_{k+1}-x_k$, $y_k=F(x_{k+1})-F(x_k)$. For a nonzero step the displayed “good Broyden” update satisfies $B_{k+1}{s}_k=y_k$, learning the Jacobian matrix action along the observed step rather than evaluating a new full Jacobian matrix.

The update also has a least-change characterization in the Frobenius norm. Write $u=y_k-B_k{s}_k$. Any correction $E$ satisfying $E{s}_k=u$ has squared row norms bounded below by $u_i^2/\Vert s_k{\Vert }^2$, by the Cauchy-Schwarz inequality. Summing these bounds gives $\Vert E{\Vert }_F^2\ge \Vert u{\Vert }^2/\Vert s_k{\Vert }^2$, with equality for $E=u{s}_k^{\mathsf{T}}/\Vert s_k{\Vert }^2$. This proves the rank-one correction is the least-change update satisfying the secant constraint. Superlinear convergence needs local regularity and suitable initialization; it is not a global consequence of this algebraic identity.^[ref]