Past exam of the mathematics course of the University of Cambridge / 2005 / ib / Paper 4 / 19D / iii / Solution

Let $x$ and $ϵ$ denote the design and error vectors, and put $P=x{x}^T/S_{xx}$. The residual vector is $r=(I-P)ϵ$, while $\hat{\beta }-\beta =x^Tϵ/S_{xx}$. Since $(I-P)x=0$,These are jointly Gaussian linear functions of $ϵ$, so the entire residual vector is independent of $\hat{\beta }$, not merely uncorrelated with it. Choose an orthonormal basis whose first vector is $x/\sqrt{S_{xx}}$. In that basis $ϵ/\sigma$ has independent standard normal coordinates, and the residual keeps exactly the remaining $n-1$ coordinates. ThereforeTogether with part (ii), this specifies the joint distribution as the product of the slope's normal law and the scaled chi-squared distribution. In particular $E{\hat{\sigma }}^2=(n-1){\sigma }^2/n$: the variance maximum-likelihood estimator is biased downward.