Past exam of the mathematics course of the University of Cambridge / 2007 / ib / Paper 4 / 19C / i / Solution

Write $S_{xx}={\sum }_i{x}_i^2$ and $\bar{Y}=n^{-1}{\sum }_i{Y}_i$. For an identifiable two-parameter linear regression assume $S_{xx}>0$; for the variance estimate and test below also assume $n>2$. The least-squares normal equations areUsing ${\sum }_i{x}_i=0$ givesThe residual sum of squares has positive-definite quadratic part $n{\alpha }^2+S_{xx}{\beta }^2$, so these equations give its unique minimum. The normal distribution of the independent errors gives the log-likelihoodFor every fixed positive ${\sigma }^2$, maximizing this over $\alpha ,\beta$ is exactly minimizing the same residual sum of squares. Thus the ordinary least squares estimators are also the maximum-likelihood estimators.

The printed hypotheses do not explicitly require $S_{xx}>0$ or $n>2$. If $S_{xx}=0$, all $x_i$ vanish and $\beta$ is unidentifiable; this is an intercept-only model with $n-1$ residual degrees of freedom. For $n=1$ its residual is identically zero, so the likelihood has no finite maximizer with positive variance. If $S_{xx}>0$ but $n=2$, the fit is saturated, leaving no residual variance estimate or Student test with positive degrees of freedom. The following nondegenerate formulas use $S_{xx}>0$ and $n>2$.