Past exam of the mathematics course of the University of Cambridge / 2017 / ii / Paper 1 / 28K (Principles of Statistics)

For observation $X=k$, the likelihood is proportional to $p^k(1-p{)}^{n-k}$. For $0<k<n$, its logarithmic derivative is $k/p-(n-k)/(1-p)$, vanishing uniquely at $k/n$; the logarithm is strictly concave. At $k=0$ or $k=n$, monotonicity gives the boundary maximum. Thus the maximum-likelihood estimator isA uniform prior multiplies this likelihood by a constant. The beta-integral normalization therefore gives the posterior distributionIn particular the posterior mean $(k+1)/(n+2)$ is not automatically the optimal estimator for the weighted loss in the next part.

for an estimator $\delta (X)$ and loss $L$, its risk function is $R(p,\delta )={\mathbb{E}}_{p}L(\delta (X),p)$, where expectation is over repeated data at fixed $p$. Its Bayes risk under prior $\pi$ is $r(\pi ,\delta )=\int R(p,\delta )\pi (dp)$, equivalently the joint expectation under the prior and sampling model.

Condition on $X=k$. For $0<k<n$, the posterior expected loss is a strictly convex quadratic in the decision $d$. Its minimizer isThe beta density gives $\mathbb{E}[1/p\mid k]=(n+1)/k$ and $\mathbb{E}[1/(1-p)\mid k]=(n+1)/(n-k)$, so $d=k/n$. If $k=0$, every $d\ne 0$ has infinite posterior expected loss because of the nonintegrable $d^2/p$ term near zero, whereas $d=0$ has finite loss expectation. The analogous argument at $k=n$ forces $d=1$. Hence the Bayes estimator isIndeed $X/n$ is unbiased and has variance $p(1-p)/n$, which cancels the loss denominator. Its uniform-prior Bayes risk is $1/n$.

The printed loss is undefined at $p=0,1$, despite the stated closed parameter interval. A convention is necessary there. The natural extended loss assigns zero to the correct endpoint decision and infinity to an incorrect one; it gives endpoint risks zero for $X/n$. Alternatively a continuous extension of this estimator's risk gives endpoint values $1/n$. The interior result and the minimax value below are unchanged by either convention, and a uniform prior gives the endpoints measure zero. We do not identify an undefined $0/0$ with a value without specifying this choice.

For every estimator, its worst-case risk dominates its average risk under the uniform prior. Since the preceding Bayes estimator minimizes that average,The estimator $X/n$ has risk exactly $1/n$ throughout the interior, so it attains this lower bound. Under either endpoint convention discussed above its supremum over $[0,1]$ is also $1/n$. ThusThis constant-risk Bayes argument works also for $n=1$, when the only observed counts are endpoints and the posterior integrability argument still selects their corresponding decisions.