Past exam of the mathematics course of the University of Cambridge / 2022 / iii / Paper 219 / 4 / a

Using Bayes theorem and the fact that the prior is proper,Thus $\hat{I}$ is an unbiased estimator of $Z^{-1}$, and the Harmonic mean estimator of Bayesian model evidence is $\hat{Z}=1/\hat{I}$. The reciprocal is not itself generally unbiased, though it is consistent when the strong law of large numbers applies.

Multiplying the Gaussian likelihood and prior and completing the square givesTherefore Normal-normal conjugacy gives

The evidence is the convolution of $N(0,{\tau }^2)$ and $N(0,{\sigma }^2)$, soPart i now gives the fully simplified expectation

Let $S={\sigma }^2+{\tau }^2$. For one posterior draw, the Gaussian quadratic-exponential moment is finite precisely when $\tau <\sigma$, and thenBecause the $m$ posterior draws are independent,For $\tau \ge \sigma$ the second moment, and hence the variance, is infinite. The Harmonic mean estimator of Bayesian model evidence is therefore unstable in the usual diffuse-prior regime: posterior sampling does not adequately control the reciprocal likelihood in the posterior tails.