Past exam of the mathematics course of the University of Cambridge / 2016 / ib / Paper 1 / 19H / b

The Rao-Blackwell theorem says that an unbiased finite-variance estimator $\delta (X)$ of a parameter function $\tau (\theta )$ can be improved using a sufficient statistic $T$. DefineBecause the conditional distribution given $T$ is parameter-independent, this is a statistic computable without knowing $\theta$. The tower property of conditional expectation gives ${\mathbb{E}}_{\theta }{\delta }^{\ast }={\mathbb{E}}_{\theta }\delta =\tau (\theta )$, so it remains unbiased. The law of total variance givesHence the improved estimator has no larger variance or mean squared error:Equality holds precisely when $\delta ={\delta }^{\ast }(T)$ almost surely under the relevant parameter. More generally, even without unbiasedness, conditional Jensen inequality shows that conditioning on a sufficient statistic cannot increase risk for any convex loss in the estimate, provided the expectations exist. The squared-loss result above is the usual variance form of the theorem.