Bayesian model evidence ($\mathcal{Z}$)

The Bayesian model evidence, also called the marginal likelihood, isIt normalizes the posterior and averages the likelihood over the prior.

A prior predictive check simulates parameters from the prior distribution and observations from their sampling model, then examines whether those observations are plausible before conditioning on data. This reveals unintended information from apparently broad priors, especially after nonlinear transformations or when scales and correlations are coupled. It differs from a posterior predictive check, which uses the posterior distribution.

Given posterior draws ${\theta }_i$, the harmonic mean estimator isAlthough the average inside brackets estimates $Z^{-1}$, its variance is often infinite because reciprocal likelihoods grow rapidly in the posterior tails.

A Bayes factor compares two models through the ratio of their model evidences, $B_{01}=p(D\mid M_0)/p(D\mid M_1)$.

For a binomial distribution count, compare a point null $\theta =1/2$ with the uniform alternative prior distribution on $(0,1)$. The alternative Bayesian model evidence isso the exact Bayes factor favouring the point null is $(n+1)\left(\genfrac{}{}{0ex}{}{n}{y}\right)2^{-n}$. With standardized deviation $z=2(y-n/2)/\sqrt{n}$, the local normal distribution approximation is $B_{01}\approx \sqrt{2n/\pi }{e}^{-z^2/2}$. At fixed $z$ the frequentist p-value remains approximately constant, while the factor grows like $\sqrt{n}$. This demonstrates the prior-width mechanism behind the Jeffreys-Lindley paradox for a Gaussian point null in a discrete sampling model.

A Gaussian practical-null mixture compares a narrow centered normal distribution prior with a wider centered normal distribution prior. Neither component is a point mass. With observed mean $y\mid \beta \sim N(\beta ,1/n)$ and prior precisions $q_0>q_1>0$, its model predictive variances are $V_i=1/n+1/q_i$, and the Bayes factor isThe overall posterior density is a Bayesian model averaging mixture with component means $ny/(n+q_i)$. A wide-prior penalty can favor the practical null near zero; its dominance depends quantitatively on both prior scales and the observation, not merely on the observation being of order $n^{-1/2}$.

For suitable nested models whose common nuisance parameters have matching priors, the Savage-Dickey density ratio expresses a Bayes factor as posterior density divided by prior density at the nested parameter value.