Past exam of the mathematics course of the University of Cambridge / 2017 / ib / Paper 3 / 20H / a

The Gauss-Markov theorem states that with full-column-rank design, mean-zero errors and covariance ${\sigma }^{2}I$, the ordinary least squares estimator is the best linear unbiased estimator: its covariance is no larger than any other such estimator's in the Loewner order. Normality is unnecessary for that theorem.

In the given normal linear model, maximizing the likelihood function is equivalent to minimizing $\Vert Y-X\beta {\Vert }^2$. The normal equations and full rank giveFor a direct variance comparison, write ${\beta }^{\ast }=AY$. Unbiasedness for every $\beta$ requires $AX=I_p$. Set $B=(X^{T}X{)}^{-1}{X}^T$ and $D=A-B$, so $DX=0$. The cross covariance terms vanish because $B{D}^T=(X^{T}X{)}^{-1}(DX{)}^T=0$. ThereforeThis proves the requested comparison for every contrast $t$ without relying only on the theorem's name.