= Solution
Let $r=Ax-b$ and consider an arbitrary variation $x+th$. The objective for the <linear least-squares problem> is
$$
\|A(x+th)-b\|^2=\|r\|^2+2t\,h^TA^Tr+t^2\|Ah\|^2.
$$
At a minimizer the <derivative> at $t=0$ must vanish for every $h$, so $A^Tr=0$. Conversely, when this holds, the same expansion at $t=1$ gives $\|A(x+h)-b\|^2=\|r\|^2+\|Ah\|^2\geq\|r\|^2$ for every $h$. Thus \b[$x$ minimizes the residual norm if and only if $A^T(Ax-b)=0$]. These <least-squares normal equations> do not require $A$ to have full rank; without full rank the minimizing vector need not be unique.
The original PDF has $A_{22}=1$, not the 2 in the converted TeX. Using the PDF's <matrix> gives
$$
A^TA=\begin{pmatrix}27&6\\6&15\end{pmatrix},\qquad A^Tb=\begin{pmatrix}10\\5\end{pmatrix},\qquad\det(A^TA)=369.
$$
The <least-squares normal equations> consequently have the unique solution
$$
\boxed{x=\frac1{123}\begin{pmatrix}40\\25\end{pmatrix}.}
$$
As a direct check, $Ax-b=(-279,-95,238,-61)^T/123$, whose <inner product> with either column of $A$ is zero. The positive <determinant> verifies full column rank for this particular problem.
Back to article page
