Past exam of the mathematics course of the University of Cambridge / 2007 / ii / Paper 3 / 26I (Principles of Statistics)

For a prior $\Pi$ and loss $L$, the Bayes risk of a rule $d$ is $r(\Pi ,d)=\int {\mathbb{E}}_{\theta }L(\theta ,d(X))\Pi (d\theta )$. A Bayes decision rule minimizes this integrated risk. An Extended Bayes rule is a rule whose excess over the optimal Bayes risk can be made arbitrarily small by a suitable proper prior: for every $\epsilon >0$ some $\Pi$ has $r(\Pi ,d)\le {\inf }_{d^{\prime }}r(\Pi ,d^{\prime })+\epsilon$, with finite risks.

Condition on the observation and use iterated expectation. A rule is Bayes precisely when it chooses a minimizer of the posterior expected loss almost surely, up to the usual measurability requirement. Under squared Euclidean loss, with a finite posterior second moment, write $m=\mathbb{E}[\theta \mid X]$. The identityshows that the Bayes rule is the posterior mean.

Write $X=\theta +Z$, with $Z\sim N(0,1)$ independent of the prior $\theta \sim N(0,{\tau }^{-1})$. Then $\lambda X-\theta =(\lambda -1)\theta +\lambda Z$, soCompleting the square in the joint density gives $\theta \mid X\sim N(X/(1+\tau ),1/(1+\tau ))$. Thus the Bayes rule is $d_{1/(1+\tau )}$, with Bayes risk $1/(1+\tau )$. The rule $d_1=X$ has integrated risk one for every $\tau >0$, so its excess Bayes risk is $\tau /(1+\tau )\to 0$ as $\tau \downarrow 0$. The priors remain proper at every positive $\tau$. Therefore $d_1$ is extended Bayes, without treating the limiting improper flat prior as a proper Bayes prior.