Past exam of the mathematics course of the University of Cambridge / 2005 / ii / Paper 3 / 5I / a / Solution

The least-squares estimator minimizes $\Vert Y-Xb{\Vert }^2$. Differentiating in $b$ gives $-2X^T(Y-Xb)=0$, hence $\boxed{X^{T}X\hat{\beta }=X^{T}Y}$. Full column rank makes $X^{T}X$ positive definite, soThis is a linear transformation of a multivariate normal distribution. Its mean is $\beta$ and its covariance is ${\sigma }^2(X^{T}X{)}^{-1}$. Therefore $\boxed{\hat{\beta }\sim N_p(\beta ,{\sigma }^2(X^{T}X{)}^{-1})}$.