Past exam of the mathematics course of the University of Cambridge / 2014 / ib / Paper 1 / 7H (Statistics)

For an estimator with finite second moment, its bias and mean squared error areWrite $\hat{\theta }-\theta =(\hat{\theta }-{\mathbb{E}}_{\theta }\hat{\theta })+b_{\theta }$. On squaring and taking expectations, the cross term vanishes. This proves the bias-variance decomposition of mean squared error:For the sample mean, independence gives $\mathbb{E}\bar{X}=\theta$, $\mathrm{Var}(\bar{X})={\theta }^2/n$. Thus $k\bar{X}$ has bias $(k-1)\theta$. Unbiasedness throughout the parameter family requiresAt the single parameter value $\theta =0$, all these estimators are zero almost surely; that special value does not change the requirement of unbiasedness for all $\theta$.

By the bias-variance decomposition of mean squared error,For $\theta \ne 0$, comparison with the unbiased estimator reduces toIts roots are $(n-1)/(n+1)$ and $1$, soEquality at either endpoint gives the same risk as $\bar{X}$. At $\theta =0$, every risk is zero, so no strict improvement is possible. This illustrates shrinking a sample mean with variance proportional to mean squared: allowing a small bias can reduce variance enough to improve total risk. The best constant is $k=n/(n+1)$, whose mean squared error is ${\theta }^2/(n+1)$.