Past exam of the statistics course of the University of Oxford / 2017 / 1 / a

For the independent statistical sample, write the likelihood function and log-likelihood function asThe expected Fisher information in the whole statistical sample isUnder the regularity conditions that permit differentiation under the integral sign, the score function has expected value zero andFor an interior statistical parameter, assuming identifiability and $0<I_1(\theta )<\infty$, the regular asymptotic normality of a maximum likelihood estimator statesThus the large-sample normal approximation is ${\hat{\theta }}_n\dot{\sim }N(\theta ,I_n(\theta {)}^{-1})$. The inverse information is the leading sampling variance approximation; the displayed convergence specifies its precise asymptotic meaning.

For $0<{\sigma }^2<\infty$, the central limit theorem givesIf $g$ is differentiable at $\mu$, the delta method applies to the first-order expansionwhere the remainder notation means that its product with $\sqrt{n}$ tends to zero in probability. HenceThe useful nondegenerate normal approximation requires $g^{\prime }(\mu )\ne 0$. When $g^{\prime }(\mu )=0$, the first-order limit is zero and a higher-order expansion may be needed.

For a Bernoulli distribution with fixed $0<p<1$, the mean and variance are $p$ and $p(1-p)$. The odds transformation $g(x)=x/(1-x)$ has derivative $g^{\prime }(x)=(1-x{)}^{-2}$. Applying the delta method from part (ii),More precisely, $\sqrt{n}\{\overline{X}/(1-\overline{X})-p/(1-p)\}$ converges to a normal distribution of mean zero and variance $p/(1-p{)}^3$, after defining the statistic arbitrarily on $\overline{X}=1$. That exceptional event has probability $p^n\to 0$, so this extension does not change the limit. The unextended ratio is undefined there and has no finite exact variance if interpreted as infinite. At $p=0$ it is zero almost surely; at $p=1$ the denominator is always zero. This is the delta method for Bernoulli odds.