Past exam of the mathematics course of the University of Cambridge / 2005 / ii / Paper 3 / 5I / b / Solution

Define the hat matrix $H=X(X^{T}X{)}^{-1}{X}^T$. It is symmetric and idempotent, the orthogonal projection onto the column space of $X$. Since $HX=X$,The covariance is ${\sigma }^{2}H{H}^T={\sigma }^{2}H$. This normal distribution can be singular when $p<n$, since all fitted vectors lie in the $p$-dimensional model subspace.