Past exam of the mathematics course of the University of Cambridge / 2007 / ii / Paper 3 / 26I / i

For a prior $\Pi$ and loss $L$, the Bayes risk of a rule $d$ is $r(\Pi ,d)=\int {\mathbb{E}}_{\theta }L(\theta ,d(X))\Pi (d\theta )$. A Bayes decision rule minimizes this integrated risk. An Extended Bayes rule is a rule whose excess over the optimal Bayes risk can be made arbitrarily small by a suitable proper prior: for every $\epsilon >0$ some $\Pi$ has $r(\Pi ,d)\le {\inf }_{d^{\prime }}r(\Pi ,d^{\prime })+\epsilon$, with finite risks.

Condition on the observation and use iterated expectation. A rule is Bayes precisely when it chooses a minimizer of the posterior expected loss almost surely, up to the usual measurability requirement. Under squared Euclidean loss, with a finite posterior second moment, write $m=\mathbb{E}[\theta \mid X]$. The identityshows that the Bayes rule is the posterior mean.