Past exam of the options I course of the University of Oxford / 2012 / O / O2

For the full-column-rank design matrix, the normal-error maximum-likelihood estimator is the least-squares estimatorIndeed maximizing the Gaussian likelihood over $\theta$ minimizes $\Vert Y-X\theta {\Vert }^2$, whose normal equations are $X^{T}X\hat{\theta }=X^{T}Y$; rank $p$ makes $X^{T}X$ invertible. Substitution of the model givesSince $\mathbb{E}ϵ=0$, it follows that $\boxed{\mathbb{E}\hat{\theta }=\theta }$, proving unbiasedness.

Independence and common error variance give $\mathrm{Var}(ϵ)={\sigma }^2{I}_n$. For the fixed matrix $A=(X^{T}X{)}^{-1}{X}^T$, the covariance transformation rule givesThus the covariance is determined by the error variance and the inverse Gram matrix of the design columns.

The design columns are $1$, $x$ and $z$. Their pairwise inner products are zero by the three supplied sums. Put $S_{xx}={\sum }_i{x}_i^2$ and $S_{zz}={\sum }_i{z}_i^2$, positive under the full-rank assumption. ThenApplying part (a),The off-diagonal covariances vanish because the design columns are orthogonal. With Gaussian errors, the coefficient estimators are also jointly normal and therefore independent.

The future mean response is $E_{\ast }=\alpha +\beta x_{\ast }+\gamma z_{\ast }$. By invariance of maximum likelihood estimation, its estimator is the fitted meanIts sampling variance is the quadratic form of the coefficient covariance matrix with $c=(1,x_{\ast },z_{\ast }{)}^T$:This estimator is unbiased because each fitted coefficient is unbiased. The future error ${ϵ}_{\ast }$ does not enter the estimator of the mean, so its independent variance is not part of this expression.