Past exam of the options II course of the University of Oxford / 2010 / O (Statistics)

The maximum-likelihood estimator is a maximizerWriting ${\ell }_n=\log L_n$ and the score $S_n={\nabla }_{\theta }{\ell }_n$, the expected Fisher information matrix iswhere the equality uses the usual regularity conditions. For independent identically distributed observations, ${\mathcal{I}}_n=n{\mathcal{I}}_1$.

Under the standard identifiability, interior-parameter, differentiability and nonsingular-information assumptions, the large-sample distribution isor approximately $\hat{\theta }\sim N(\theta ,{\mathcal{I}}_n(\theta {)}^{-1})$. Boundary parameters need not obey this regular approximation.

The two totals are independent, with $n\sim \mathrm{Bin}(2N,q)$ and $m\sim \mathrm{Bin}(2M,p)$. Their joint likelihood isDifferentiating the separated log-likelihood givesThese formulas include all-zero or all-one totals as boundary maxima. For interior $p,q$, the Fisher information in the ordered parameter vector $(p,q)$ isThe off-diagonal entries vanish because the likelihood separates, and the two estimates are independent even at finite sample size. For large $M,N$, their joint distribution is approximately

A linear combination of independent normal variables is normal; the corresponding asymptotic statement follows from the joint central limit theorem and the continuous mapping theorem. ThusThe two variances add because of independence. This approximation assumes both sample sizes grow and the true probabilities are interior.

Under the null, replace both parameters by a common $r$. The likelihood becomesThe pooled maximum-likelihood estimate isUnder an interior null $p=q=r$, part (ii) gives a centered asymptotic normal difference with variance $r(1-r)(1/(2M)+1/(2N))$. The law of large numbers makes $\hat{r}\to r$ in probability. By Slutsky's theorem, replacing $r$ by this consistent estimate givesFor the two-sided alternative, reject when $|Z|>z_{1-\alpha /2}$, the corresponding standard-normal quantile. This has asymptotic significance level $\alpha$.

For a two-sided 5% test, the critical values are $\pm 1.96$. Since $|-1.89|<1.96$, do not reject the null at the 5% level.

For the prespecified directional alternative $q>p$, the numerator $\hat{p}-\hat{q}$ tends to be negative. Use the lower-tail critical region $Z<-1.64$ at the 5% level. Since $-1.89<-1.64$, reject in favor of $q>p$ in the one-sided test. The direction should be chosen from the additional scientific information rather than selected after inspecting the sign of this same statistic.

The significance level of a test is a bound $\alpha$ on its probability of rejecting a true null; for a composite null the bound is uniform over its parameter values. Its power at an alternative parameter is its rejection probability under that parameter.

The Neyman-Pearson lemma concerns a simple null density $f_0$ and a simple alternative density $f_1$ for the whole sample. Among tests of size at most $\alpha$, a most powerful test rejects where $f_1>k{f}_0$, accepts where $f_1<k{f}_0$, and if necessary randomizes on equality so that its size is $\alpha$. Let its rejection probability function be ${\varphi }_{\ast }$, with $0\le {\varphi }_{\ast }\le 1$, and let $\varphi$ be any competing test of size at most $\alpha$.

Pointwise,on the rejection side both factors are nonnegative, and on the acceptance side both are nonpositive. Integrating givesThus the likelihood-ratio test has at least as much power as any competitor. Even when the sample has a continuous density, the likelihood ratio can have an atom, so boundary randomization may still be necessary in the general statement. For the exponential test below the statistic has a continuous distribution and no randomization is needed.

For a fixed alternative $0<\lambda <{\lambda }_0$, the joint likelihood ratio isIt is strictly increasing in $S={\sum }_i{x}_i$. By the Neyman-Pearson lemma, the most powerful level-$\alpha$ rejection region is therefore $S\ge c_{\alpha }$. Under the null, $S\sim \mathrm{Gamma}(n,{\lambda }_0)$ with the second parameter interpreted as a rate. Its continuous distribution givesThis threshold depends on the null and $\alpha$, but not on the particular smaller alternative $\lambda$. Thus the same test is most powerful for every $0<\lambda <{\lambda }_0$, making it uniformly most powerful.

Under a true alternative $\lambda$, $S\sim \mathrm{Gamma}(n,\lambda )$, so its attained power isFor each $\lambda <{\lambda }_0$, the Neyman-Pearson lemma bounds the power of every other level-$\alpha$ test by this value. Hence the displayed function is not only an upper bound but the attainable power envelope on the specified alternative.