Past exam of the mathematics course of the University of Cambridge / 2013 / iii / Paper 30 / 1 / c / Solution

The ordinary least squares estimator is $\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}Y$. Consequently the fitted values and regression residuals areAn affine transformation of a multivariate normal distribution is again multivariate normal, possibly with a singular covariance matrix. For a random vector with covariance matrix $\Sigma$, its transformed covariance matrix is $A\Sigma A^T$. Since $PX=X$, $P^2=P=P^T$ and $(I-P{)}^2=I-P$, these results giveBoth multivariate normal distributions are supported on their respective projected subspaces. In particular, ${\hat{Y}}_i$ has variance ${\sigma }^2{P}_{ii}$ and $e_i$ has variance ${\sigma }^2(1-P_{ii})$; the regression residuals need not be mutually independent.