= Unique minimizer of a positive-definite quadratic function
{title2=$f(x)=\tfrac12x^TAx-b^Tx+c,\quad x_*=A^{-1}b$}
Let $A$ be a real <positive-definite matrix> and minimize the displayed <quadratic function> over all of $\mathbb R^n$. Its <gradient> is $Ax-b$, so its only <stationary point> solves the <linear system> $Ax_*=b$. The <matrix> is an <invertible matrix>, making this solution exist and be unique. <Completing the square> around that solution proves the stronger global statement:
$$
f(x)=c-\frac12b^TA^{-1}b+\frac12(x-x_*)^TA(x-x_*),\qquad f(x)-f(x_*)>0\quad(x\ne x_*).
$$
The <quadratic function> is <strictly convex>, and its <level sets> above the minimum are translated <ellipsoids>. Positivity therefore turns solving a <linear system> into locating the bottom of a bowl.<Convex Optimization by Stephen Boyd and Lieven Vandenberghe>[§3.1.4, Example 3.2, p. 71: a quadratic function is strictly convex exactly when its symmetric coefficient matrix is positive definite]{ref}
For the opening <matrix> $A=\begin{pmatrix}3&1\\1&3\end{pmatrix}$, take $b=(1,1)^T$ and $c=0$. Then $A^{-1}=\frac18\begin{pmatrix}3&-1\\-1&3\end{pmatrix}$, $x_*=(1/4,1/4)^T$, and $f(x_*)=-1/4$. Every displacement $h$ raises the value by $\tfrac12h^TAh$; along the unit <eigenvectors> $e_+$ and $e_-$ the increases are respectively $2s^2$ and $s^2$ for displacement $se_+$ or $se_-$.
For a singular <positive semidefinite matrix>, a solution of $Ax=b$ is still a global minimizer, but adding any <kernel> <vector> gives another. If $b$ is outside the <image of a linear map> defined by $A$, some <kernel> <vector> $v$ has $b^Tv\ne0$, and $f(sv)=c-sb^Tv$ is unbounded below. For an <indefinite quadratic form>, a negative <eigenvalue> gives a direction along which the quadratic term tends to $-\infty$ regardless of $b$. Thus a <stationary point> alone does not establish a minimum. The same energy-difference identity extends to the <quadratic variational principle for a symmetric positive operator>, with additional existence hypotheses in infinite <dimension>.
Back to article page
