Past exam of the options II course of the University of Oxford / 2006 / O (Statistics)

Minimize $\Vert Y-X\theta {\Vert }^2$. Differentiation gives the normal equations $X^{T}X\hat{\theta }=X^{T}Y$. Full column rank makes $X^{T}X$ positive definite, soThe expectation uses the zero error mean; the covariance uses $\mathrm{Cov}(ϵ)={\sigma }^{2}I$. With independent Gaussian errors of common variance, maximizing likelihood minimizes exactly the same residual sum of squares, so this is also the maximum likelihood estimator, whether the common variance is known or estimated.

For the quadratic design, decompose each group's residual sum of squares into its within-group constant plus the group size times the squared residual of its sample mean. The three parameter combinations $\alpha$, $\alpha +\beta +\gamma$ and $\alpha -\beta +\gamma$ can fit all three group means exactly. ThereforeThe group means are independent normal variables, with variances ${\sigma }^2/m,{\sigma }^2/n,{\sigma }^2/n$. Hence their two stated linear combinations are jointly normal, andJoint normality therefore proves the two estimators are independent. Their variances are ${\sigma }^2/(2n)$ and ${\sigma }^2/(2n)+{\sigma }^2/m$, respectively. Equal replication at the two off-center temperatures is what cancels the covariance.

A possibly randomized statistical test has rejection probability $\phi (x)\in [0,1]$ after observing $x$. Its statistical power at parameter $\theta$ is ${\mathbb{E}}_{\theta }\phi$, and its test size is the supremum of that statistical power over the null parameter set.

For simple hypotheses with densities $f_0,f_1$, the Neyman-Pearson lemma states that a level-$\alpha$ test is most powerful if it rejects where $f_1>k{f}_0$, accepts where $f_1<k{f}_0$, and randomizes on equality to obtain null rejection probability $\alpha$. Here $k\ge 0$ is chosen to reach that test size. To prove optimality, let ${\phi }_{\ast }$ be this test and $\phi$ any test of level at most $\alpha$. Pointwise,Integrating and using $\int ({\phi }_{\ast }-\phi )f_0\ge 0$ gives ${\mathbb{E}}_1{\phi }_{\ast }-{\mathbb{E}}_1\phi \ge 0$. This proves maximal statistical power. For $k>0$, equality also forces any equally powerful test to use all its allowed test size and agree with the indicated decisions away from the likelihood-ratio tie set, up to null sets.

For the Poisson sample, $S={\sum }_i{X}_i\sim \mathrm{Poisson}(n\mu )$ and, for every fixed ${\mu }_1>{\mu }_0>0$, the likelihood ratio isIt increases strictly with $S$. Choose an integer $k$ so that ${\mathbb{P}}_{{\mu }_0}(S>k)\le \alpha \le {\mathbb{P}}_{{\mu }_0}(S\ge k)$, and reject for $S>k$, with boundary rejection probabilityThis has exact test size $\alpha$. Its rejection ordering and calibration do not depend on which ${\mu }_1>{\mu }_0$ is selected, so the lemma proves it uniformly most powerful for the whole one-sided alternative. Randomization is generally needed for an exact arbitrary test size in a discrete model.

For the approximate design, the central limit theorem under ${\mu }_0=1$ gives critical sum $n+1.645\sqrt{n}$. Under ${\mu }_1$, the sum is approximately normal with mean $n{\mu }_1$ and variance $n{\mu }_1$. Target statistical power $0.9$ therefore givesor $\sqrt{n}({\mu }_1-1)\approx 1.645+1.282\sqrt{{\mu }_1}$. Thus the requested design expression isRound upward to obtain an integer design. This is a normal approximation to the requested statistical power, rather than an exact discrete sample-size guarantee; the exact Poisson test above provides its exact calibration.