Past exam of the options II course of the University of Oxford / 2006 / O / 4

Minimize $\Vert Y-X\theta {\Vert }^2$. Differentiation gives the normal equations $X^{T}X\hat{\theta }=X^{T}Y$. Full column rank makes $X^{T}X$ positive definite, soThe expectation uses the zero error mean; the covariance uses $\mathrm{Cov}(ϵ)={\sigma }^{2}I$. With independent Gaussian errors of common variance, maximizing likelihood minimizes exactly the same residual sum of squares, so this is also the maximum likelihood estimator, whether the common variance is known or estimated.

For the quadratic design, decompose each group's residual sum of squares into its within-group constant plus the group size times the squared residual of its sample mean. The three parameter combinations $\alpha$, $\alpha +\beta +\gamma$ and $\alpha -\beta +\gamma$ can fit all three group means exactly. ThereforeThe group means are independent normal variables, with variances ${\sigma }^2/m,{\sigma }^2/n,{\sigma }^2/n$. Hence their two stated linear combinations are jointly normal, andJoint normality therefore proves the two estimators are independent. Their variances are ${\sigma }^2/(2n)$ and ${\sigma }^2/(2n)+{\sigma }^2/m$, respectively. Equal replication at the two off-center temperatures is what cancels the covariance.