= Solution
Because $X$ has <full column rank>, $X^TX$ is invertible. The <ordinary least squares> estimator is
$$
\boxed{\widehat\beta=(X^TX)^{-1}X^TY}.
$$
It satisfies the <normal equations>
$$
X^T(Y-X\widehat\beta)=0,
$$
so the residual is orthogonal to the <column space> of $X$. For any $\beta$,
$$
Y-X\beta
=(Y-X\widehat\beta)+X(\widehat\beta-\beta).
$$
The two terms are orthogonal, and the <Pythagorean theorem in an inner-product space> gives
$$
S(\beta)
=S(\widehat\beta)
+(\beta-\widehat\beta)^TX^TX(\beta-\widehat\beta)
\ge S(\widehat\beta).
$$
\b[Thus $\widehat\beta$ minimizes the least-squares objective.]
Back to article page
