= Solution
Minimizing the sum of squared residuals gives the <normal equations for linear least squares>, $X^TX\widehat\theta=X^TY$. Full column rank makes $X^TX$ positive definite and invertible, so the unique minimizer is
$$
\boxed{\widehat\theta=AY,\qquad A=(X^TX)^{-1}X^T}.
$$
Because $\mathbb EY=X\theta$ and $\operatorname{Cov}(Y)=\sigma^2I_n$, one has $AX=I_p$ and
$$
\mathbb E\widehat\theta=\theta,\qquad
\operatorname{Cov}(\widehat\theta)=\sigma^2AA^T
=\boxed{\sigma^2(X^TX)^{-1}}.
$$
For the alternative linear estimator the corresponding general formulas are
$$
\boxed{\mathbb E(DY)=DX\theta,\qquad\operatorname{Cov}(DY)=\sigma^2DD^T}.
$$
No normality assumption on the errors is needed for these claims.
Back to article page
