Past exam of the options I course of the University of Oxford / 2012 / O / O1

Let ${\ell }_n(\theta )={\sum }_{i=1}^n\log f(X_i;\theta )$ be the log-likelihood. Under the usual regularity conditions, the expected Fisher information in the whole sample isThe equality of the two expressions uses mean-zero score and permitted differentiation of the normalization identity. Independent observations make the information additive. The maximum-likelihood estimator has asymptotic normality:The factor $n$ is included in $I_n$; it must not be inserted a second time into the variance.

Let $z={\Phi }^{-1}(1-\alpha /2)$ for the standard normal cumulative distribution function. Consistency of the maximum-likelihood estimator and Slutsky's theorem give the asymptotic standardizationIts probability of lying between $-z$ and $z$ tends to $1-\alpha$. Solving those inequalities for the unknown parameter yields the Wald confidence intervalThus the estimated standard error is $I_n(\hat{\theta }{)}^{-1/2}$. Its coverage is approximate and relies on the large-sample regularity assumptions.

For the stated geometric distribution, write $S={\sum }_i{X}_i$. ThenSince $\mathbb{E}{X}_i=1/\theta$, we have $\mathbb{E}(S-n)=n(1-\theta )/\theta$. Taking expectation gives the sample Fisher informationThe support begins at one, so this expectation uses the trials-until-first-success convention of the geometric law.

For the Bernoulli distribution, with $S={\sum }_i{X}_i$,Using $\mathbb{E}S=n\theta$, the expected Fisher information is

For the same sample size $n$, the geometric distribution provides more Fisher information:The corresponding asymptotic variances of the maximum-likelihood estimators areHence choose the geometric sample for more precise inference at equal numbers of observations. Its asymptotic variance is $\theta$ times as large, and its standard error is $\sqrt{\theta }$ times as large. This illustrates Fisher information per observation and per trial. It is a per-observation comparison: one geometric observation records a waiting time containing, on average, $1/\theta$ Bernoulli trials. A constraint on the total number of underlying trials would be a different sampling-budget comparison.