= Harmonic mean estimator of Bayesian model evidence
{c}
Let $\pi$ be a proper <prior density>, let $L>0$ for prior-almost every parameter, and let $Z=\int L\pi\in(0,\infty)$. For posterior draws $\theta_i$, the harmonic mean estimator is
$$
\widehat Z=\left[\frac1m\sum_{i=1}^mL(\theta_i)^{-1}\right]^{-1}.
$$
The identity $\mathbb E_{\pi(\cdot\mid y)}[1/L]=1/Z$ follows by cancelling $L$ in the posterior density. The variance of $1/L$ is often infinite, making the estimate unstable even when an ergodic average is consistent.<Bayes Factors by Robert E. Kass and Adrian E. Raftery>[Section 4.3, pp. 779–780, equation (11) and reciprocal-likelihood instability]{ref}
The support qualification matters. If $a=\int_{\{L>0\}}\pi<1$, cancellation instead gives $a/Z$. For a uniform prior on $[0,1]$ and $L(\theta)=\mathbf1_{[0,1/2]}(\theta)$, the evidence is $1/2$, but every posterior likelihood value is one and the displayed estimator is always one. An improper prior also leaves the absolute evidence normalization undefined.
Back to article page
