Past exam of the mathematics course of the University of Cambridge / 2005 / ib / Paper 4 / 19D (Statistics)

Assume $S_{xx}={\sum }_i{x}_i^2>0$, which is necessary to identify the slope in linear regression through the origin. The joint normal likelihood has logarithm, apart from a constant,Complete the square in $\beta$. With $\hat{\beta }={\sum }_i{x}_i{Y}_i/S_{xx}$ and $\mathrm{RSS}={\sum }_i(Y_i-\hat{\beta }{x}_i{)}^2$,The maximum likelihood estimation therefore first minimizes at $\hat{\beta }$; differentiating in ${\sigma }^2$ then maximizes atThis is a genuine finite maximum when $\mathrm{RSS}>0$. If all $x_i=0$, the slope is unidentifiable. If $\mathrm{RSS}=0$, the likelihood is unbounded as ${\sigma }^2\downarrow 0$, so no positive-variance maximum exists. For nonzero design and $n\ge 2$, zero residual has probability zero under the stated positive-variance model.

The least-squares estimator satisfiesA linear combination of independent normal random variables is normal; its mean is $\beta$ and its variance is ${\sigma }^2{\sum }_i{x}_i^2/S_{xx}^2$. Hence

Let $x$ and $ϵ$ denote the design and error vectors, and put $P=x{x}^T/S_{xx}$. The residual vector is $r=(I-P)ϵ$, while $\hat{\beta }-\beta =x^Tϵ/S_{xx}$. Since $(I-P)x=0$,These are jointly Gaussian linear functions of $ϵ$, so the entire residual vector is independent of $\hat{\beta }$, not merely uncorrelated with it. Choose an orthonormal basis whose first vector is $x/\sqrt{S_{xx}}$. In that basis $ϵ/\sigma$ has independent standard normal coordinates, and the residual keeps exactly the remaining $n-1$ coordinates. ThereforeTogether with part (ii), this specifies the joint distribution as the product of the slope's normal law and the scaled chi-squared distribution. In particular $E{\hat{\sigma }}^2=(n-1){\sigma }^2/n$: the variance maximum-likelihood estimator is biased downward.

For $n\ge 2$, use the unbiased residual variance $s^2=\mathrm{RSS}/(n-1)=n{\hat{\sigma }}^2/(n-1)$. Under the null hypothesis, $(\hat{\beta }-{\beta }_0)\sqrt{S_{xx}}/\sigma$ is standard normal and independent of $\mathrm{RSS}/{\sigma }^2\sim {\chi }_{n-1}^2$. HenceFor a chosen significance level $\alpha$, reject the null in favor of the two-sided alternative exactly when $|T|>t_{n-1,1-\alpha /2}$, the stated upper quantile of the Student t-distribution. The exact two-sided p-value is $2P(t_{n-1}\ge |T_{\mathrm{obs}}|)$. This hypothesis test does not substitute the biased MLE for $s^2$ without correcting its factor and does not use a normal critical value when variance is unknown.