Past exam of the mathematics course of the University of Cambridge / 2010 / ii / Paper 3 / 5J / Solution

Assume $n>p\ge 1$, the ordinary nonsaturated Gaussian linear model. The log likelihood is, up to a constant,The normal equations and full column rank give the least-squares estimator $\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}Y$. Write $H=X(X^{T}X{)}^{-1}{X}^T$ and $Q=I-H$, an orthogonal projection of rank $n-p$. Maximizing in ${\sigma }^2$ yieldsSince $QX=0$, the residual sum of squares is ${\epsilon }^{T}Q\epsilon$. Expanding its expectation uses only $\mathbb{E}{\epsilon }_i{\epsilon }_j={\sigma }^2{\delta }_{ij}$:This does not use Cochran's theorem. It proves bias $-p{\sigma }^2/n$ and givesIn the saturated case $n=p$, residuals vanish identically, the likelihood is unbounded as the positive variance tends to zero, and the unbiased residual estimator is unavailable. Thus the usual variance-estimation request requires $n>p$.