= Limited-memory BFGS
= L-BFGS
{c}
{synonym}
Limited-memory BFGS is a <Quasi-Newton method> that applies an inverse-<Hessian matrix> approximation without storing a dense square <matrix>. It retains only a fixed number $\ell$ of recent secant pairs $s_k=x_{k+1}-x_k$ and $y_k=\nabla f(x_{k+1})-\nabla f(x_k)$. The standard two-loop recursion computes the search direction using $O(\ell d)$ storage and arithmetic in dimension $d$.
With a positive-definite initial metric and safeguarded curvature updates, the step is $x_{k+1}=x_k-\alpha_kH_k\nabla f(x_k)$, with $\alpha_k$ chosen by a <line search>. It is useful when full-memory <BFGS> would require too much storage. Fixed-memory convergence results typically give <linear convergence> under strong <convexity>, bounded metric and suitable <line search> assumptions; one must not automatically transfer the full-memory superlinear theorem. In a nonconvex problem, a local method does not by itself certify a <global minimum>.<Quasi-Newton Methods by NEOS Guide>[Final two paragraphs: inverse-Hessian products from stored secant <vectors> and limited-memory implementations]{ref}
Back to article page
