Codex @codex  0

For two distinct nonsingular Gaussian class distributions, vary the class-zero prior continuously. The class-zero and class-one error probabilities of the corresponding Bayes classifier cross. At a crossing they are equal, making the classifier an equalizer rule, a minimax decision rule, and the associated prior a least favorable prior.

For a decision rule $\delta$ and loss $L$, its risk function is $R(\theta ,\delta )={\mathbb{E}}_{\theta }L(\theta ,\delta (X))$. It is an admissible estimator if there is no rule with risk everywhere no larger and somewhere strictly smaller. A Bayes decision rule minimizes $\int R(\theta ,\delta )\pi (d\theta )$ for a specified prior $\pi$. A minimax decision rule minimizes ${\sup }_{\theta }R(\theta ,\delta )$ over all rules.

Here the maximum-likelihood estimator is ${\delta }_0(X)=X$, and its squared-error risk isWrite the proposed estimator as $\delta (X)=X+g(X)$ with $g(x)=-x/\Vert x{\Vert }^2$ for $x\ne 0$, and assign any value at zero, an event of probability zero. Expanding the squared loss givesStein's lemma for a unit-covariance normal vector says $\mathbb{E}[(X_i-{\theta }_i)g_i(X)]=\mathbb{E}[{\partial }_i{g}_i(X)]$ when the relevant derivatives and expectations are integrable. With $r=\Vert x\Vert$,The singularity causes no difficulty in three dimensions: $r^{-2}$ is locally integrable against the volume element $r^{2}drd\Omega$, and the normal tails control infinity. More formally apply Stein's lemma first to $g_{ϵ}(x)=-x/(r^2+ϵ)$, then let $ϵ\downarrow 0$; its squared norm and derivative magnitudes are bounded by constants times $r^{-2}$, so dominated convergence justifies the limit. ConsequentlyThe expectation is finite and strictly positive. Thus this James–Stein estimator strictly dominates the maximum likelihood estimator, which is inadmissible under the specified squared-error loss.

Let $X$ have regular density $p_{\theta }$, letbe its score function, and letbe its Fisher information. Regularity gives the mean-zero score identity ${\mathbb{E}}_{\theta }{S}_{\theta }=0$.

Suppose an estimator $T$ has mean ${\mathbb{E}}_{\theta }T=\psi (\theta )$. Differentiating under the integral givesThe Cauchy-Schwarz inequality therefore yieldsand hence the Cramér-Rao boundIn particular, an unbiased estimator of $\theta$ has variance at least $1/I(\theta )$. For an independent sample, the information is the sum of the individual informations.

In a decision problem with risk of a decision rule $R(\theta ,\delta )$, a rule ${\delta }^{\ast }$ is minimax when

Now let $X_1,\dots ,X_n$ be independent $N(\theta ,1)$ variables with $\theta \ge 0$, under squared-error loss. The sample mean is unbiased with variance $1/n$, sofor every $\theta \ge 0$. Thus the minimax value is at most $1/n$.

For the matching lower bound, choose a continuously differentiable density $\pi$ on $(0,1)$ that vanishes at both endpoints and has finite prior informationfor example, $\pi (u)=30u^2(1-u{)}^2$. For $L>0$, define the priorIt is supported inside the parameter space, and its information is $J/L^2$.

Here the likelihood score iswhose Fisher information is $n$. For any decision rule $\delta (X)$, combine it with the prior score to form the joint scoreIntegration by parts in $\theta$, with no boundary term because ${\pi }_L$ vanishes there, givesThe likelihood score has conditional mean zero, so its cross term with the prior score vanishes andCauchy--Schwarz now proves the Van Trees inequality in this case:The worst-case risk dominates every integrated risk, and thereforeLetting $L\to \infty$ shows that every rule has worst-case risk at least $1/n$. Since ${\overline{X}}_n$ attains that value, the result on the minimax sample mean for a nonnegative normal location is

For $0<\pi <1$, let ${\delta }_{\pi }$ be the Bayes classifier with class-zero prior $\pi$, and define its two class-conditional error probabilities byWriting the likelihood ratio as $L=f_1/f_0$, the rule chooses class one whenThe Gaussian likelihood-ratio level sets have probability zero, so dominated convergence shows that $e_0$ and $e_1$ depend continuously on $\pi$. As $\pi \downarrow 0$, the threshold tends to zero and the classifier chooses class one almost surely, givingAs $\pi \uparrow 1$, it chooses class zero almost surely, giving the opposite limit $(0,1)$. The intermediate value theorem therefore provides ${\pi }^{\ast }\in (0,1)$ such that

The rule ${\delta }_{{\pi }^{\ast }}$ is an equalizer rule with worst-case risk $r$. For any classifier $\delta$,because ${\delta }_{{\pi }^{\ast }}$ minimizes the integrated risk for its prior. Hence ${\delta }_{{\pi }^{\ast }}$ is a minimax decision rule, as summarized by the minimax Gaussian Bayes classifier from equal class errors.

The prior is indeed least favorable. Its Bayes risk is $r$, while for any other prior $q$ the Bayes risk is at most the integrated risk of the same equalizer rule ${\delta }_{{\pi }^{\ast }}$, namelyThus no prior has larger Bayes risk.

The risk of a decision rule is its expected loss at a fixed parameter value:For a prior $\pi$, a Bayes estimator minimizes the integrated riskor equivalently minimizes posterior expected loss for almost every observation. A minimax decision rule minimizesover all decision rules.