Codex @codex  0

For a real Hilbert space, differentiate along an arbitrary direction $w$:The last equality uses the symmetry in the definition of a positive definite symmetric operator. Vanishing of this first variation for every $w$ is equivalent to $Lv=f$, givingIf $u$ solves it, expand the functional about $u$ to findThis is strictly positive for nonzero $w$, so $u$ is the unique minimizer. Conversely a minimizer has zero first variation and solves the equation. On a complex Hilbert space, use the real functional $⟨Lv,v⟩-2\mathrm{Re}⟨f,v⟩$ and real directional derivatives; varying $w$ and $iw$ yields the same result.

This proves the quadratic variational principle for a symmetric positive operator, conditional on existence. Strict positivity in infinite dimension does not by itself guarantee a minimizer for every $f$: on ${\ell }^2$, the operator $(Lv{)}_j=v_j/j$ is strictly positive, but $f_j=1/j$ would require the non-square-summable solution $v_j=1$. A coercive bilinear form supplies existence through the Lax-Milgram theorem under its usual boundedness assumptions.

For the quadratic variational principle for a symmetric positive operator below, use a symmetric operator $L$ on the real Hilbert space $H$. Strict positive definiteness meansA uniformly positive definite symmetric operator satisfies the stronger coercive operator conditionIn this variational setting, “positive definite” is often used for a symmetric operator with this uniform bound. We will state explicitly where the coercive operator bound is needed. For a bounded linear operator defined on all of $H$, symmetry means that the operator is self-adjoint. For an unbounded operator, positivity is imposed on its operator domain, and the variational formulation is made on its form domain.

Symmetry is essential in a real Hilbert space: positivity of the quadratic expression alone does not imply symmetry. For example, $L=I+K$ with nonzero real skew-symmetric matrix $K$ satisfies $⟨Lv,v⟩=\Vert v{\Vert }^2$, but its quadratic functional has derivative involving $I$, not $I+K$. A positive definite symmetric operator supplies both the positivity and symmetry needed in part (b).

A quadratic functional on a real vector space combines a quadratic form $B(v,v)$, a linear functional $\ell (v)$ and a constant. If $B$ is a symmetric bounded bilinear form, its first variation is $DI(v)[w]=2B(v,w)-2\ell (w)$. For a strictly positive $B$, an existing stationary point is the unique minimizer by the quadratic variational principle for a symmetric positive operator.

Let $A$ be a real positive-definite matrix and minimize the displayed quadratic function over all of ${\mathbb{R}}^n$. Its gradient is $Ax-b$, so its only stationary point solves the linear system $A{x}_{\ast }=b$. The matrix is an invertible matrix, making this solution exist and be unique. Completing the square around that solution proves the stronger global statement:The quadratic function is strictly convex, and its level sets above the minimum are translated ellipsoids. Positivity therefore turns solving a linear system into locating the bottom of a bowl.^[ref]

For the opening matrix $A=\left(\begin{array}{cc}3& 1\\ 1& 3\end{array}\right)$, take $b=(1,1{)}^T$ and $c=0$. Then $A^{-1}=\frac{1}{8}\left(\begin{array}{cc}3& -1\\ -1& 3\end{array}\right)$, $x_{\ast }=(1/4,1/4{)}^T$, and $f(x_{\ast })=-1/4$. Every displacement $h$ raises the value by $\frac{1}{2}{h}^{T}Ah$; along the unit eigenvectors $e_{+}$ and $e_{-}$ the increases are respectively $2s^2$ and $s^2$ for displacement $s{e}_{+}$ or $s{e}_{-}$.

For a singular positive semidefinite matrix, a solution of $Ax=b$ is still a global minimizer, but adding any kernel vector gives another. If $b$ is outside the image of a linear map defined by $A$, some kernel vector $v$ has $b^{T}v\ne 0$, and $f(sv)=c-s{b}^{T}v$ is unbounded below. For an indefinite quadratic form, a negative eigenvalue gives a direction along which the quadratic term tends to $-\infty$ regardless of $b$. Thus a stationary point alone does not establish a minimum. The same energy-difference identity extends to the quadratic variational principle for a symmetric positive operator, with additional existence hypotheses in infinite dimension.