Past exam of the mathematics course of the University of Cambridge / 2010 / ii / Paper 2 / 5J (Statistical Modelling)

A maximum-likelihood estimator maximizes the likelihood as a function of the parameter for the observed data. Under the usual regularity conditions, a scalar maximum-likelihood estimator satisfies $\sqrt{n}(\hat{\theta }-\theta )\Rightarrow N(0,I_1(\theta {)}^{-1})$, where $I_1$ is the Fisher information in one observation.

For a Poisson distribution with $\theta >0$, the log-likelihood is $\ell (\theta )={\sum }_j{Y}_j\log \theta -n\theta -{\sum }_j\log (Y_j!)$. Solving ${\ell }^{\prime }(\theta )=0$ gives $\boxed{\hat{\theta }=\bar{Y}}$. Its expectation is $\theta$ and its variance is $\theta /n$. If all observations are zero, the maximum over $\theta \ge 0$ occurs at zero; over $\theta >0$ this is a boundary supremum. The one-observation score function is $Y/\theta -1$, so $I_1(\theta )=\mathrm{Var}(Y)/{\theta }^2=1/\theta$ and the asymptotic variance of $\sqrt{n}(\bar{Y}-\theta )$ is $\theta$.

For $n\ge 2$, let $S=(n-1{)}^{-1}{\sum }_j(Y_j-\bar{Y}{)}^2$. The identitygives $\mathbb{E}S=(n\theta -\theta )/(n-1)=\theta$, so $S$ is another unbiased estimator. However, the total score function has variance $n/\theta$. For any regular unbiased estimator $T$, differentiating ${\mathbb{E}}_{\theta }T=\theta$ gives $\mathrm{Cov}(T,{\ell }^{\prime })=1$, so the Cauchy-Schwarz inequality yields $\mathrm{Var}(T)\ge \theta /n$. The sample mean attains this Cramér-Rao bound exactly. There is no variance advantage in preferring $S$ to $\bar{Y}$.