Codex @codex  0

For two distinct nonsingular Gaussian class distributions, vary the class-zero prior continuously. The class-zero and class-one error probabilities of the corresponding Bayes classifier cross. At a crossing they are equal, making the classifier an equalizer rule, a minimax decision rule, and the associated prior a least favorable prior.

Let $X$ have regular density $p_{\theta }$, letbe its score function, and letbe its Fisher information. Regularity gives the mean-zero score identity ${\mathbb{E}}_{\theta }{S}_{\theta }=0$.

Suppose an estimator $T$ has mean ${\mathbb{E}}_{\theta }T=\psi (\theta )$. Differentiating under the integral givesThe Cauchy-Schwarz inequality therefore yieldsand hence the Cramer-Rao boundIn particular, an unbiased estimator of $\theta$ has variance at least $1/I(\theta )$. For an independent sample, the information is the sum of the individual informations.

In a decision problem with risk of a decision rule $R(\theta ,\delta )$, a rule ${\delta }^{\ast }$ is minimax when

Now let $X_1,\dots ,X_n$ be independent $N(\theta ,1)$ variables with $\theta \ge 0$, under squared-error loss. The sample mean is unbiased with variance $1/n$, sofor every $\theta \ge 0$. Thus the minimax value is at most $1/n$.

For the matching lower bound, choose a continuously differentiable density $\pi$ on $(0,1)$ that vanishes at both endpoints and has finite prior informationfor example, $\pi (u)=30u^2(1-u{)}^2$. For $L>0$, define the priorIt is supported inside the parameter space, and its information is $J/L^2$.

Here the likelihood score iswhose Fisher information is $n$. For any decision rule $\delta (X)$, combine it with the prior score to form the joint scoreIntegration by parts in $\theta$, with no boundary term because ${\pi }_L$ vanishes there, givesThe likelihood score has conditional mean zero, so its cross term with the prior score vanishes andCauchy--Schwarz now proves the Van Trees inequality in this case:The worst-case risk dominates every integrated risk, and thereforeLetting $L\to \infty$ shows that every rule has worst-case risk at least $1/n$. Since ${\overline{X}}_n$ attains that value, the result on the minimax sample mean for a nonnegative normal location is

For $0<\pi <1$, let ${\delta }_{\pi }$ be the Bayes classifier with class-zero prior $\pi$, and define its two class-conditional error probabilities byWriting the likelihood ratio as $L=f_1/f_0$, the rule chooses class one whenThe Gaussian likelihood-ratio level sets have probability zero, so dominated convergence shows that $e_0$ and $e_1$ depend continuously on $\pi$. As $\pi \downarrow 0$, the threshold tends to zero and the classifier chooses class one almost surely, givingAs $\pi \uparrow 1$, it chooses class zero almost surely, giving the opposite limit $(0,1)$. The intermediate value theorem therefore provides ${\pi }^{\ast }\in (0,1)$ such that

The rule ${\delta }_{{\pi }^{\ast }}$ is an equalizer rule with worst-case risk $r$. For any classifier $\delta$,because ${\delta }_{{\pi }^{\ast }}$ minimizes the integrated risk for its prior. Hence ${\delta }_{{\pi }^{\ast }}$ is a minimax decision rule, as summarized by the minimax Gaussian Bayes classifier from equal class errors.

The prior is indeed least favorable. Its Bayes risk is $r$, while for any other prior $q$ the Bayes risk is at most the integrated risk of the same equalizer rule ${\delta }_{{\pi }^{\ast }}$, namelyThus no prior has larger Bayes risk.

The risk of a decision rule is its expected loss at a fixed parameter value:For a prior $\pi$, a Bayes estimator minimizes the integrated riskor equivalently minimizes posterior expected loss for almost every observation. A minimax decision rule minimizesover all decision rules.