Past exam of the mathematics course of the University of Cambridge / 2019 / ii / Paper 4 / 28J (Principles of Statistics)

For observations $x=(x_1,\dots ,x_n)$, the likelihood function isA maximum-likelihood estimator is any measurable choiceFor one observation in a regular scalar model, the score function isand the Fisher information iswhere the second identity requires the usual differentiation-under-the-integral regularity. For $n$ independent observations, Tensorization of Fisher information gives $I_n(\theta )=nI(\theta )$.

The strong law of large numbers givesA continuous one-to-one real function is strictly monotone and has a continuous inverse on its image. Hencewhenever ${\hat{\mu }}_n$ lies in that image; an arbitrary extension outside the image handles the remaining finite-sample values. The continuous mapping theorem gives ${\hat{\theta }}_n\to {\theta }_0$ almost surely, so this is a consistent estimator.

PutThe central limit theorem gives $\sqrt{n}({\hat{\mu }}_n-\mu ({\theta }_0))\Rightarrow N(0,{\sigma }_h^2)$. Provided ${\mu }^{\prime }({\theta }_0)\ne 0$, the delta method applied to ${\mu }^{-1}$ yieldsStrict monotonicity alone does not exclude ${\mu }^{\prime }({\theta }_0)=0$; in that exceptional case this root-$n$ limit need not hold and higher-order analysis is required.

One does not necessarily have $\mathrm{Var}({\hat{\theta }}_n)\ge [nI({\theta }_0){]}^{-1}$ for every $n$, because ${\hat{\theta }}_n$ need not be unbiased. The Cramer-Rao bound for an estimator with bias $B_n(\theta )$ is insteadunder the corresponding regularity assumptions.

For each $i$, let ${\hat{\theta }}_{(-i)}$ be the same estimator computed after deleting $X_i$, and put ${\overline{\theta }}_{(-)}=n^{-1}{\sum }_i{\hat{\theta }}_{(-i)}$. The jackknife bias correction isIts bias isso the leading $a/n$ term has been removed and the bias is smaller than that of ${\hat{\theta }}_n$.

The assumed maximum-likelihood estimator is the sample mean of $h(X_i)$. Its central limit theorem isFor a regular model, the regular maximum-likelihood limit isThe two asymptotic variances must agree, soConsequentlyto first asymptotic order. The estimator therefore attains the Cramer-Rao bound asymptotically: its precision, the reciprocal variance, exhausts the information in the sample.