Past exam of the mathematics course of the University of Cambridge / 2007 / ib / Paper 1 / 18C (Statistics)

Let $K={\sum }_i{X}_i$ and $\bar{X}=K/n$. The likelihood is ${\theta }^K(1-\theta {)}^{n-K}$, so for $0<K<n$ its unique maximizer is $\hat{\theta }=K/n$. By invariance of maximum likelihood estimation,For $K=0$ or $n$, the strict parameter interval $(0,1)$ has no attained likelihood maximum; the usual extended estimate takes the limiting endpoint and gives $\hat{\varphi }=0$. Thus the displayed estimator is the conventional extended MLE on those samples, an important boundary qualification. This is the first estimator in estimating Bernoulli variance.

Independence gives $\mathbb{E}[X_1(1-X_2)]=\theta (1-\theta )=\varphi$, proving unbiasedness. Conditional on $K=k$, all binary sample strings with $k$ ones are equally likely. ThereforeThe Rao-Blackwell theorem, or the tower property and total variance decomposition directly, yields the improved estimatorIt is unbiased by conditional expectation and is a function of the requested MLE. Its variance is strictly smaller, sinceFor $0<\theta <1$, the event $K=1$ has positive probability. On it, $X_1(1-X_2)$ has probability $1/n$ of being one, strictly between zero and one for $n\ge 2$. Thus the final expectation is strictly positive. This proves strict variance reduction, not merely a weak inequality.

The uniform prior and likelihood give the posterior densitya Beta distribution with parameters $K+1,n-K+1$. Under squared-error loss for $\varphi$, the posterior expected loss is $\mathbb{E}[{\varphi }^2\mid K]-2a\mathbb{E}[\varphi \mid K]+a^2$, minimized uniquely at the posterior mean of $\varphi$, not at the product of the two posterior mean factors.

Using the supplied beta integral,Thus the Bayes estimate isThis remains positive even on the endpoint samples, unlike the extended MLE; it integrates over the posterior uncertainty in $\theta$.