Past exam of the mathematics course of the University of Cambridge / 2018 / ib / Paper 1 / 19H / a

Minimizing $\Vert Y-X\theta {\Vert }^2$ gives the normal equations $X^{T}X\hat{\theta }=X^{T}Y$. Full column rank makes $X^{T}X$ invertible, soWith the hat matrix $H=X(X^{T}X{)}^{-1}{X}^T$, the residual is $\hat{\epsilon }=(I-H)\epsilon$. The symmetric projection $I-H$ has rank $n-p$, henceand the latter estimator is unbiased.