= BFGS method
{c}
= BFGS
{c}
{synonym}
A <Quasi-Newton method> that maintains a symmetric curvature approximation for differentiable <optimization>. For a <positive-definite symmetric matrix> $H$ approximating inverse curvature, step $s$, gradient change $y$ and $\rho=1/(y^{\mathsf T}s)>0$, its inverse update is
$$
H_+=(I-\rho sy^{\mathsf T})H(I-\rho ys^{\mathsf T})+\rho ss^{\mathsf T}.
$$
Multiplying by $y$ gives $H_+y=s$, the inverse form of the <Quasi-Newton secant equation>. The update is a <symmetric matrix>. For any nonzero $v$,
$$
v^{\mathsf T}H_+v=[(I-\rho ys^{\mathsf T})v]^{\mathsf T}H[(I-\rho ys^{\mathsf T})v]+\rho(s^{\mathsf T}v)^2>0.
$$
Both terms are nonnegative, and both can vanish only if $s^{\mathsf T}v=0$ and then $v=0$. Thus positive definiteness is preserved when $y^{\mathsf T}s>0$, making $-H_+\nabla f$ a <descent direction> away from a stationary point. Full updates use quadratic storage in the dimension; the method does not require explicit <Hessian matrix> evaluations. <Superlinear convergence> requires additional local smoothness and <line search> assumptions.<Quasi-Newton Methods by NEOS Guide>[Secant condition and low-rank curvature updating; BFGS is the zero-parameter member of the displayed family.]{ref}
Back to article page
