= Matrix-free Hessian-vector product
{title2=$v\mapsto\nabla^2l(x)v$}
Applying a <Hessian matrix> to a <vector> without explicitly storing all its entries. For $H=A^{\mathsf T}\Delta A$ with diagonal $\Delta$, evaluate $Hv=A^{\mathsf T}(\Delta(Av))$. This uses two matrix-vector multiplications and a diagonal scaling: $O(nd)$ arithmetic for dense $A\in\mathbb R^{n\times d}$, rather than forming the $d\times d$ <Hessian matrix>. Iterative second-order methods can use this operation, with safeguards for indefinite curvature when the objective is nonconvex.
Back to article page
