= Solution
Minimizing $\|Y-X\theta\|^2$ gives the <normal equations> $X^TX\widehat\theta=X^TY$. Full column rank makes $X^TX$ invertible, so
$$
\boxed{\widehat\theta=(X^TX)^{-1}X^TY.}
$$
With the <hat matrix> $H=X(X^TX)^{-1}X^T$, the residual is $\widehat\varepsilon=(I-H)\varepsilon$. The symmetric projection $I-H$ has rank $n-p$, hence
$$
\boxed{\frac{\widehat\varepsilon^T\widehat\varepsilon}{\sigma^2}
\sim\chi^2_{n-p},\qquad
\widehat{\sigma}^2=\frac{\widehat\varepsilon^T\widehat\varepsilon}{n-p}}
$$
and the latter estimator is unbiased.
Back to article page
