Past exam of the Probability, statistics and options II course of the University of Oxford / 2010 / O (Statistics)

The likelihood is $L(\theta ;x)={\prod }_{i=1}^{n}f(x_i;\theta )$, and a maximum likelihood estimator is any maximizer $\hat{\theta }\in \arg {\max }_{\theta }L(\theta ;X)$. Write $\ell =\log L$ and score $U={\nabla }_{\theta }\ell$. Fisher information for the sample iswhere the equality uses the usual differentiability and interchange assumptions. For independent identically distributed observations $I_n=n{I}_1$. Under standard interior, identifiability and regularity assumptions,Without those assumptions the large-sample normal assertion need not hold, for example at a boundary parameter.

The totals are independent binomials, $n\sim \mathrm{Bin}(2N,q)$ and $m\sim \mathrm{Bin}(2M,p)$. ThusDifferentiating the log likelihood, or maximizing the two factors separately, givesThis remains the constrained maximum at zero or one when a total is at an endpoint. At interior parameters the sample information, in coordinate order $(p,q)$, isFor example $-\mathbb{E}{\ell }_{pp}=\mathbb{E}[m/p^2+(2M-m)/(1-p{)}^2]=2M/[p(1-p)]$; the cross derivative is zero. As both sample sizes grow with nondegenerate interior probabilities, the joint approximation is normal with mean $(p,q)$ and covariance $\mathrm{diag}(p(1-p)/(2M),q(1-q)/(2N))$. The independence of the two estimators is exact, inherited from the two data samples.

A linear combination of independent normal random variables is normal, with variances adding when the coefficients are $1,-1$. Applying the joint central limit approximation from (i) givesEquivalently the difference minus $p-q$, divided by the displayed standard deviation, converges in distribution to a standard normal under the same regular large-sample regime.

Under the null put $p=q=r$. The likelihood becomes the same two binomial constants times $r^{m+n}(1-r{)}^{2M+2N-m-n}$. Maximization gives the pooled estimateUnder an interior null, (ii) has zero mean and variance $r(1-r)(1/(2M)+1/(2N))$. Statistical consistency of $\hat{r}$ and Slutsky's theorem permit replacement of $r$ by $\hat{r}$ in the denominator, giving the stated $Z\Rightarrow N(0,1)$. For a two-sided level-$\alpha$ test, reject when $\boxed{|Z|>{\Phi }^{-1}(1-\alpha /2)}$. The large-sample approximation requires both expected counts of each type to grow; if all observed alleles are identical the stated denominator degenerates and this approximation is not applicable.

At level $0.05$ the two-sided critical values are $\pm 1.96$. Since $|-1.89|<1.96$, do not reject the two-sided null. The approximate two-sided $p$-value is $2\Phi (-1.89)\approx 0.059$.

For the specified alternative $q>p$, the statistic $\hat{p}-\hat{q}$ should be negative. Reject in the lower tail, $Z<-1.64$. Since $-1.89<-1.64$, reject for this one-sided alternative, with approximate $p$-value $\Phi (-1.89)\approx 0.029$. The direction of the tail follows the order of subtraction in the statistic.

For a rejection function $\phi (x)\in [0,1]$, its power is $\beta (\theta )={\mathbb{E}}_{\theta }\phi (X)$. A test has level at most $\alpha$ if ${\sup }_{\theta \in H_0}\beta (\theta )\le \alpha$; its size is that supremum. In a simple null the significance probability is just the rejection probability under that null.

For two simple hypotheses with joint sample densities $f_0,f_1$, the Neyman–Pearson lemma states that a test rejecting where $f_1>k{f}_0$, accepting where $f_1<k{f}_0$, and randomizing if necessary on equality to have size $\alpha$, is most powerful among level-$\alpha$ tests. Let ${\phi }_{\ast }$ be that test and $\phi$ any competitor. Pointwise,Integration givessince $k\ge 0$, the first test has size $\alpha$, and the other has size at most $\alpha$. This proves the claimed power bound, allowing null-density-zero regions as well. Equality implies agreement away from the likelihood-ratio boundary, up to null sets, when the inequalities are strict.

For a fixed $\lambda <{\lambda }_0$, the likelihood ratio against the null isstrictly increasing in $S={\sum }_i{x}_i$. The Neyman–Pearson critical region is therefore an upper tail in $S$, and its null threshold depends only on the desired size, not on which $\lambda <{\lambda }_0$ was chosen. Thus the same test is uniformly most powerful for the entire one-sided alternative.

Under the null $S\sim \mathrm{Gamma}(n,{\lambda }_0)$ with the second parameter interpreted as rate. Its continuous distribution permits the exact thresholdUnder any fixed alternative its rejection probability is $\boxed{Q(\lambda )=1-F(c_{\alpha };n,\lambda )}$. Applying the lemma separately to each such $\lambda$ shows that every level-$\alpha$ test has power at most this value there. The constructed test attains the bound for all $\lambda <{\lambda }_0$ simultaneously.