Past exam of the mathematics course of the University of Cambridge / 2017 / ii / Paper 3 / 26K (Principles of Statistics)

The score function isWith common parameter-independent support and justified differentiation under the integral, ${\mathbb{E}}_{\theta }[{\partial }_{\theta }\log f(X_1,\theta )]=\int {\partial }_{\theta }f(x,\theta )dx={\partial }_{\theta }1=0$. Linearity then gives $\boxed{{\mathbb{E}}_{{\theta }_0}{S}_n({\theta }_0)=0}$. These regularity conditions exclude, for example, moving-support uniform models.

The per-observation Fisher information is $I(\theta )={\mathbb{E}}_{\theta }[({\partial }_{\theta }\log f(X_1,\theta ){)}^2]$. Sinceand the integral of $f_{\theta \theta }$ is the second derivative of $1$, the regularity assumptions giveFor independent observations the total Fisher information is $nI(\theta )$, because centered individual score functions have zero cross terms.

A maximum-likelihood estimate ${\hat{\theta }}_n$ is a measurable maximizer of ${\ell }_n(\theta )={\sum }_i\log f(X_i,\theta )$ over $\Theta$. Under identifiability, consistency conditions, an interior true parameter, sufficient differentiability and finite positive Fisher information, the standard limits areThe variance in the normal distribution is the reciprocal of the per-observation Fisher information, not of $nI$. Strong consistency needs additional hypotheses if it is to replace convergence in probability; the regular asymptotic result stated here does not require claiming it.

The almost-sure bound gives ${\tilde{\theta }}_n-{\hat{\theta }}_n=o_{\mathbb{P}}(n^{-1/2})$, so Slutsky theorem transfers both consistency and the asymptotic normal law to ${\tilde{\theta }}_n$. The continuous mapping theorem and delta method then giveIf ${\psi }^{\prime }({\theta }_0)=0$, the latter is the Dirac measure at zero, conventionally a degenerate normal distribution; a nondegenerate second-order limit would need a different normalization and extra smoothness.