Past exam of the mathematics course of the University of Cambridge / 2012 / iii / Paper 36 / 3

Fix any estimator $\hat{\eta }$ and put $A(x)=d(\hat{\eta }(x),f)$, $B(x)=d(\hat{\eta }(x),g)$ and $D=d(f,g)$. The triangle inequality for the metric gives $A+B\ge D$. HenceThe larger of the two risk functions dominates their average. Using nonnegativity to replace both probability density functions by their minimum,The right side is independent of the estimator, so taking the infimum provesThis metric squared-loss two-point bound uses overlap of the observation laws to quantify how hard it is to distinguish the two separated parameters.

Apply the preceding argument to the two product observation laws and the real-valued estimator $T={\hat{\psi }}_n$. Alternatively, directly useThe average of the two mean squared errors is at least $(\theta -\tau {)}^2/4$ times the overlap of the two joint probability density functions. Their overlap equalsTaking the infimum over estimators therefore yieldsHere $R_n^{\ast }$ is the minimax risk for this functional. The factor $\frac{1}{2}\Vert P_f^{(n)}-P_g^{(n)}{\Vert }_1$ is the total variation distance, so the same metric squared-loss two-point bound applies to a functional even when distinct density parameters have the same functional value.

For each $n\ge 1$, choose the continuous probability density functionsThe second is at least $1/2$ and integrates to one. Their expected values differ byWriting $g_n/f=1+{\Delta }_n$ givesThe supplied product bound, also obtained from the chi-squared divergence of product measures, impliesFor the final elementary estimate, compare the exponential series with the geometric series: $e^x\le (1-x{)}^{-1}$ for $0\le x<1$. Thus the overlap of the product laws is at least $1/2$. Part (b) now gives the explicit mean-estimation minimax lower bound for continuous densitiesSo one may take $C=1/1152$, uniformly for all $n\ge 1$. The alternatives are allowed to depend on $n$, because the minimax risk takes a supremum over all continuous probability density functions separately at each sample size.