Past exam of the options I course of the University of Oxford / 2012 / O / O2 / a

For the full-column-rank design matrix, the normal-error maximum-likelihood estimator is the least-squares estimatorIndeed maximizing the Gaussian likelihood over $\theta$ minimizes $\Vert Y-X\theta {\Vert }^2$, whose normal equations are $X^{T}X\hat{\theta }=X^{T}Y$; rank $p$ makes $X^{T}X$ invertible. Substitution of the model givesSince $\mathbb{E}ϵ=0$, it follows that $\boxed{\mathbb{E}\hat{\theta }=\theta }$, proving unbiasedness.

Independence and common error variance give $\mathrm{Var}(ϵ)={\sigma }^2{I}_n$. For the fixed matrix $A=(X^{T}X{)}^{-1}{X}^T$, the covariance transformation rule givesThus the covariance is determined by the error variance and the inverse Gram matrix of the design columns.