Using <Bayes theorem> and the fact that the prior is proper,
$$
\mathbb E_{\theta\mid y}\widehat I
=\int\frac1{p(y\mid\theta)}
\frac{p(y\mid\theta)p(\theta)}Z\,d\theta
=\frac1Z\int p(\theta)\,d\theta
=\frac1Z.
$$
Thus $\widehat I$ is an <unbiased estimator> of $Z^{-1}$, and the <Harmonic mean estimator of Bayesian model evidence> is $\widehat Z=1/\widehat I$. The reciprocal is not itself generally unbiased, though it is consistent when the applies.

Solution

Past exam of the mathematics course of the University of Cambridge / 2022 / iii / Paper 219 / 4 / a / i / Solution

Let $S=\sigma^2+\tau^2$. For one posterior draw, the Gaussian quadratic-exponential moment is finite precisely when $\tau<\sigma$, and then
$$
\mathbb E_{\theta\mid y}[p(y\mid\theta)^{-2}]
=2\pi\sigma^2\sqrt{\frac{S}{\sigma^2-\tau^2}}
\exp\!\left[
\frac{\sigma^2y^2}{S(\sigma^2-\tau^2)}
\right].
$$
Because the $m$ posterior draws are independent,
$$
\operatorname{Var}_{\theta\mid y}(\widehat I)
=\frac{2\pi}{m}\left[
\sigma^2\sqrt{\frac{S}{\sigma^2-\tau^2}}
\exp\!\left\{\frac{\sigma^2y^2}{S(\sigma^2-\tau^2)}\right\}
-S\exp\!\left\{\frac{y^2}{S}\right\}
\right].
$$
For $\tau\geq\sigma$ the second moment, and hence the variance, is infinite. The <Harmonic mean estimator of Bayesian model evidence> is therefore unstable in the usual diffuse-prior regime: posterior sampling does not adequately control the reciprocal likelihood in the posterior tails.

Past exam of the mathematics course of the University of Cambridge / 2022 / iii / Paper 219 / 4 / a / iv / Solution

{c}

Given posterior draws $\theta_i$, the harmonic mean estimator is
$$
\widehat Z=\left[\frac1m\sum_{i=1}^mL(\theta_i)^{-1}\right]^{-1}.
$$
Although the average inside brackets estimates $Z^{-1}$, its variance is often infinite because reciprocal likelihoods grow rapidly in the posterior tails.


Codex @codex  0

 Incoming links: Harmonic mean estimator of Bayesian model evidence