Rao-Blackwell theorem

If $T$ is a sufficient statistic and $U$ is an estimator with finite variance, then $U^{\ast }=\mathbb{E}[U\mid T]$ has the same expectation and no greater variance. Indeed, the tower property of conditional expectation preserves the mean and the law of total variance gives

For independent uniform random variables on $(\theta ,3\theta )$ with $\theta >0$, the sample minimum and maximum form a sufficient statistic. Conditional on these endpoints, exchangeability and the uniform law of the remaining observations give $E[X_1\mid X_{(1)},X_{(n)}]=(X_{(1)}+X_{(n)})/2$. Applying the Rao-Blackwell theorem to $X_1/2$ gives the displayed unbiased estimator. Its variance is ${\theta }^2/[2(n+1)(n+2)]$, compared with ${\theta }^2/12$ for $X_1/2$.

For independent geometric observations with mass $p^{r-1}(1-p)$, the total $T$ is sufficient. Conditional on $T=t$, positive-integer compositions are uniformly distributed. The indicator $1_{\{X_1>1\}}$ is unbiased for $p$, and conditioning it on $T$ gives $\left(\genfrac{}{}{0ex}{}{t-2}{n-1}\right)/\left(\genfrac{}{}{0ex}{}{t-1}{n-1}\right)=(t-n)/(t-1)$. For one observation, use $1_{\{T>1\}}$ instead, including its zero value at $T=1$.

An unbiased estimator that is a function of a complete sufficient statistic is the unique uniformly minimum-variance unbiased estimator, up to almost sure equality. Rao-Blackwellization supplies variance reduction, and completeness supplies uniqueness. The theorem applies to a conditional statistical experiment as well, when unbiasedness, sufficiency and completeness are all evaluated under that conditional family.

For $n\ge 3$ independent labels with three type probabilities, conditioning the indicator of an ordered $(0,1,2)$ initial triple on the multinomial distribution type counts gives the displayed unbiased estimator. The Rao-Blackwell theorem reduces its variance strictly when all three probabilities are positive. At a boundary parameter with a missing type, both variances vanish.

Replace an estimator by its conditional expectation given selected information. The mean is unchanged and the variance cannot increase, by the law of total variance. In simulation this produces conditional Monte Carlo estimators. Sufficiency is needed for the classical inferential strengthening, but not for the elementary variance reduction identity.

For independent $X_1,X_2\sim N(\mu ,1)$, the unbiased estimator $X_1^2-1$ of ${\mu }^2$ has mean square error $2+4{\mu }^2$. Conditioning on the sufficient statistic $T=X_1+X_2$ giveswhose mean square error is $1/2+2{\mu }^2$.