Past exam of the mathematics course of the University of Cambridge / 2011 / ii / Paper 1 / 5J / Solution

Write $u(y,\theta )={\partial }_{\theta }\log f(y,\theta )$ for the single-observation score function. Differentiating the normalization of the probability density function givesFor discrete observations, use a sum instead. The allowed interchange of differentiation and integration is the usual regular-model assumption with common support. Adding these identities gives ${\mathbb{E}}_{\theta }{U}_n=0$.

Let $p_{\theta }$ be the joint density. Because $T$ is an unbiased estimator with no explicit dependence on the parameter, differentiating ${\mathbb{E}}_{\theta }T=\theta$ givesThe scores are independent random variables with mean zero, so variance additivity for independent random variables gives ${\mathrm{Var}}_{\theta }(U_n)=n{\mathrm{Var}}_{\theta }(U_1)$. The Cauchy-Schwarz inequality applied to the centred estimator and the score now gives the Cramér-Rao lower bound:The denominator is the sample Fisher information. This bound concerns regular models with positive finite information; the preceding covariance identity explains why a small estimator variance requires a sufficiently informative score.