Hessian of a sum of functions of linear forms (${\nabla }^{2}l(x)=A^{\mathsf{T}}\mathrm{diag}(y_i{f}^{\prime \prime }(a_i^{\mathsf{T}}x))A$)

For twice differentiable $f:\mathbb{R}\to \mathbb{R}$, constant vectors $a_i$ and constant scalars $y_i$, let $l(x)={\sum }_i{y}_{i}f(a_i^{\mathsf{T}}x)$, and let row $i$ of $A$ be $a_i^{\mathsf{T}}$. The chain rule givesThe second formula is the displayed matrix product because entry $(j,k)$ is the same weighted sum ${\sum }_i{y}_i{f}^{\prime \prime }(a_i^{\mathsf{T}}x)a_{ij}{a}_{ik}$. Signed diagonal factors do not ensure a positive-definite Hessian matrix. The factorization also gives an efficient matrix-free Hessian-vector product.