Rao-Blackwell theorem

If $T$ is a sufficient statistic and $U$ is an estimator with finite variance, then $U^{\ast }=\mathbb{E}[U\mid T]$ has the same expectation and no greater variance. Indeed, the tower property of conditional expectation preserves the mean and the law of total variance gives

An unbiased estimator that is a function of a complete sufficient statistic is the unique uniformly minimum-variance unbiased estimator, up to almost sure equality. Rao-Blackwellization supplies variance reduction, and completeness supplies uniqueness. The theorem applies to a conditional statistical experiment as well, when unbiasedness, sufficiency and completeness are all evaluated under that conditional family.

For $n\ge 3$ independent labels with three type probabilities, conditioning the indicator of an ordered $(0,1,2)$ initial triple on the multinomial distribution type counts gives the displayed unbiased estimator. The Rao-Blackwell theorem reduces its variance strictly when all three probabilities are positive. At a boundary parameter with a missing type, both variances vanish.

Replace an estimator by its conditional expectation given selected information. The mean is unchanged and the variance cannot increase, by the law of total variance. In simulation this produces conditional Monte Carlo estimators. Sufficiency is needed for the classical inferential strengthening, but not for the elementary variance reduction identity.

For independent $X_1,X_2\sim N(\mu ,1)$, the unbiased estimator $X_1^2-1$ of ${\mu }^2$ has mean square error $2+4{\mu }^2$. Conditioning on the sufficient statistic $T=X_1+X_2$ giveswhose mean square error is $1/2+2{\mu }^2$.