Past exam of the mathematics course of the University of Cambridge / 2016 / ib / Paper 4 / 19H (Statistics)

Let $Q={\sum }_i{x}_i^2>0$. Under the usual nondegenerate regression assumption $n>2$, maximizing the normal likelihood is equivalent to minimizing the residual sum of squares. The normal equations and ${\sum }_i{x}_i=0$ give the maximum-likelihood estimatorsWrite $A=\overline{\epsilon }$ and $B={\sum }_i{x}_i{\epsilon }_i/Q$. Then $\hat{\alpha }=\alpha +A$, $\hat{\beta }=\beta +B$, and the residual is $r_i={\epsilon }_i-A-x_{i}B$. Independence of the errors givesFor each $i$,Thus the three requested variables are pairwise uncorrelated.

They are linear functions of the jointly normal error vector. More strongly, the whole vector $(A,B,r_1,\dots ,r_n)$ has a multivariate normal distribution, and its block covariances show that $A$, $B$ and the residual vector are independent, by uncorrelated jointly normal variables are independent. Independence is preserved when applying a measurable function to the residual block. HenceThe residual projection is $I-11^{\mathsf{T}}/n-x{x}^{\mathsf{T}}/Q$, of rank $n-2$, so additionally ${\sum }_i{r}_i^2/{\sigma }^2\sim {\chi }_{n-2}^2$. This is the orthogonal projection of a Gaussian vector interpretation of the same result. The printed design conditions alone allow $n=2$; then the fit is exact, the positive-variance likelihood has no finite maximizer, and the residual test in part (b) is undefined. The standard testing question therefore needs $n>2$.

Put ${\mathrm{RSS}}_1={\sum }_i(Y_i-\hat{\alpha }-\hat{\beta }{x}_i{)}^2$ for the unrestricted fit. Under $H_0$, the best slope is still $\hat{\beta }$ because the design is centered. Orthogonality of the residuals to the constant vector givesAt either fitted mean, the maximum-likelihood estimator of variance is $\mathrm{RSS}/n$. The ratio used by the generalized likelihood-ratio test is consequentlySmall likelihood ratios correspond to large $|T_n|$. Under $H_0$, part (a) gives independent $\sqrt{n}\overline{Y}/\sigma \sim N(0,1)$ and ${\mathrm{RSS}}_1/{\sigma }^2\sim {\chi }_{n-2}^2$. Thus $T_n$ has the Student's t-distribution with $n-2$ degrees of freedom. The exact size-5% critical region iswhere $t_{\nu ,0.975}$ is the 97.5th percentile of that distribution. The denominator uses the unbiased residual variance estimate, not the likelihood variance denominator $n$.

Under an alternative, the same statistic has the representationThe joint law of $Z,W$ is parameter-free, and $\beta$ has disappeared. Thereforewhere $g(d)=\mathbb{P}(|(Z+\sqrt{n}d)/\sqrt{W/(n-2)}|>t_{n-2,0.975})$. This is also the noncentral t-distribution description of the power. As explained in part (a), an exact residual-based test requires $n>2$.