Past exam of the options II course of the University of Oxford / 2005 / O / 4

Minimize the residual sum of squares $\Vert Y-X\theta {\Vert }^2$. Differentiating gives the normal equations $X^{T}X\hat{\theta }=X^{T}Y$. Full column rank makes $X^{T}X$ positive definite and invertible, so the unique minimizer isSince $\mathbb{E}Y=X\theta$, it is unbiased. The independent common-variance errors give $\mathrm{Cov}(Y)={\sigma }^{2}I$, henceIndependent Gaussian errors with common variance are a sufficient condition for this estimator also to be the maximum likelihood estimator: their log-likelihood differs from $-\Vert Y-X\theta {\Vert }^2/(2{\sigma }^2)$ by terms independent of $\theta$. Profiling an unknown common variance gives the same minimizer. Mean and variance assumptions alone do not specify a likelihood.

For the seven design points, $\bar{x}=0$ and $\overline{x^2}=4$. In increasing time order the design matrix isIts columns are mutually orthogonal: the odd sums vanish and $\sum (x_i^2-4)=0$. Their squared lengths are $7,28,84$, respectively. ThereforeIn particular $\hat{\gamma }={\sum }_i(x_i^2-4)Y_i/84$. Testing for a quadratic contribution means testing $H_0:\gamma =0$ against $\gamma \ne 0$.

The cooling measurements are stated to have zero-mean errors and known common variance, but normality is not explicitly assumed for them. Under the independent-error linear regression model as stated, Chebyshev's inequality gives a valid conservative level-$\delta$ test: reject whenIts null rejection probability is at most $\delta$ because the estimator has mean zero and the calculated variance. If Gaussian errors are additionally assumed, the sharper exact test uses $Z=\sqrt{84}\hat{\gamma }/\sigma \sim N(0,1)$ under the null and rejects for $|Z|>z_{1-\delta /2}$. With only seven observations, exact normal calibration cannot be deduced from the printed moment assumptions alone.