Past exam of the mathematics course of the University of Cambridge / 2015 / ii / Paper 2 / 25J (Principles of Statistics)

For $n\ge 1$, the binomial log-likelihood is $X\log \theta +(n-X)\log (1-\theta )$ plus a constant. Its maximizer, including endpoints, is ${\hat{\theta }}_{\mathrm{MLE}}=X/n$. For $0<\theta <1$, the Fisher information isThis follows either from the expected negative second derivative or the variance of $(X-n\theta )/(\theta (1-\theta ))$. At $0,1$ the usual regular score formula does not apply; the interior information diverges as an endpoint is approached.

For a symmetric Beta distribution prior $\mathrm{Beta}(s,s)$, conjugacy gives $\theta \mid X\sim \mathrm{Beta}(X+s,n-X+s)$, with posterior mean $(X+s)/(n+2s)$. All three priors below are instances of this calculation.

The uniform prior has $s=1$, so the posterior is $\mathrm{Beta}(X+1,n-X+1)$. Its mean and mode areThus the uniform-prior posterior mode is the maximum-likelihood estimator, including $X=0,n$. The mean shrinks toward $1/2$ and equals the maximum-likelihood estimator only when $X=n/2$.

The Jeffreys prior is proportional to $[\theta (1-\theta ){]}^{-1/2}$, hence is $\mathrm{Beta}(1/2,1/2)$. Its posterior is $\mathrm{Beta}(X+1/2,n-X+1/2)$, givingThe interior case is used only when $n\ge 2$. Generally neither rule equals $X/n$: the mean agrees only at $X=n/2$; the mode agrees at the endpoints and, in the interior, at $X=n/2$. For the small experiments $n=1,2$, all possible observations happen to be among these cases, so the whole mode rule then coincides with the maximum-likelihood estimator.

Put $s=\sqrt{n}/2$. The posterior is $\mathrm{Beta}(X+s,n-X+s)$, andThe denominator is positive for $n\ge 1$. For $n>4$, both parameters exceed one and the mode simplifies to $(X+s-1)/(n+\sqrt{n}-2)$. For $n=4$ this is the uniform-prior mode, exactly $X/n$; for $n=1,2$ the endpoint or symmetric cases also make the mode rule coincide with $X/n$. Otherwise it is generally a different rule. The posterior mean agrees with $X/n$ only at $X=n/2$.

To find the minimax estimator under quadratic loss, consider the posterior mean ${\delta }_s=(X+s)/(n+2s)$. Its risk function isFor $s^2=n/4$, the numerator is identically $n/4$, soThis constant-risk binomial proportion estimator is the unique Bayes estimator for its proper beta prior under quadratic loss. A competing rule's maximum risk is at least its risk averaged over that prior, which is at least this Bayes risk. The constant-risk rule attains the bound, so the posterior mean in (iii) is the unique minimax rule. Consequently any coincident rule is minimax too: the mean in (i) is the same rule when $n=4$, and the mean in (ii) is the same rule when $n=1$. No posterior mode listed here equals that mean rule for every observation, since at $X=0$ its value is positive whereas their endpoint modes differ from it. This accounts for the small-$n$ coincidences as well as the general answer.