= Bayes factor
{c}
{title2=$B_{01}$}
{wiki}
A <Bayes factor> measures how data change the relative support for two specified <statistical models>. Write $m_j(D)=p(D\mid H_j)$ for their <Bayesian model evidence> and define the direction explicitly:
$$
\mathrm{BF}_{10}(D)=\frac{m_1(D)}{m_0(D)}.
$$
For positive finite $m_0(D),m_1(D)$, a value above one favors $H_1$ over $H_0$; a value below one favors $H_0$ over $H_1$. Reversing the comparison gives $B_{01}=\mathrm{BF}_{01}=1/\mathrm{BF}_{10}$, the opposite direction used in some formulas below.
For mutually exclusive, exhaustive $H_0,H_1$ with positive <prior probabilities>, divide their two <Bayes' theorem> formulas. Their common data normalizer cancels, leaving
$$
\underbrace{\frac{\mathbb P(H_1\mid D)}{\mathbb P(H_0\mid D)}}_{\text{posterior odds}}
=\underbrace{\frac{m_1(D)}{m_0(D)}}_{\mathrm{BF}_{10}}
\underbrace{\frac{\mathbb P(H_1)}{\mathbb P(H_0)}}_{\text{prior odds}}.
$$
The <Bayes factor> therefore multiplies <prior odds> into <posterior odds>. It is evidence conditional on the specified <statistical models> and their within-model <prior distributions>; it is not $\mathbb P(H_1\mid D)$. Model <prior probabilities> supply a separate input.<Bayes Factors by Robert E. Kass and Adrian E. Raftery>[Kass and Raftery, Bayes Factors (1995), section 3.1, p. 776: posterior odds, prior odds, and direction of the Bayes factor]{ref}
For an original stipulated example, consider ten <independent and identically distributed> <Bernoulli trials>, with heads as success. Compare $H_0:p=0.4$ with $H_1:p=0.7$, both fixed before seeing the data. Take $D$ to be one particular ordered sequence containing eight heads and two tails. The <likelihood function> under either point model is $p^8(1-p)^2$, so
$$
\mathrm{BF}_{10}
=\frac{0.7^8\,0.3^2}{0.4^8\,0.6^2}
=\left(\frac{0.7}{0.4}\right)^8\left(\frac{0.3}{0.6}\right)^2
=\frac{5764801}{262144}\approx21.9909706.
$$
If $\mathbb P(H_1)=0.1$, the <prior odds> for $H_1$ against $H_0$ are $1/9$. Hence the <posterior odds> are $21.9909706/9\approx2.4434412$, and
$$
\mathbb P(H_1\mid D)
=\frac{\mathrm{BF}_{10}}{9+\mathrm{BF}_{10}}
=\frac{5764801}{8124097}\approx0.7095928.
$$
Equal model <prior probabilities> would instead give $\mathrm{BF}_{10}/(1+\mathrm{BF}_{10})\approx0.956505$. The same <Bayes factor> can thus accompany different <posterior probabilities>. These statements condition on the two-model comparison; they do not establish that either <statistical model> describes a real coin.
\Image[/bayes-factor-heads-and-odds.png]
{title=Evidence across heads counts and its multiplication of prior odds}
{description=Original calculated <Bernoulli trials> example, with ten trials and fixed point models $H_0:p=0.4$, $H_1:p=0.7$. Left: the <Bayes factor> for each possible heads count; the highlighted eight-head result favors $H_1$. Right: at that fixed result, the <posterior odds> equal $21.9909706$ times the <prior odds>. The marked prior $\mathbb P(H_1)=0.1$ becomes <posterior probability> $0.7095928$. No empirical observations are plotted.}
Using the heads count $K=8$ instead of the ordered sequence changes the <sample space>. The <binomial distribution> assigns it mass $\binom{10}{8}p^8(1-p)^2$. The same <binomial coefficient> multiplies both models' <Bayesian model evidence> and cancels in their <Bayes factor>. Mixing an ordered-sequence numerator with a count denominator would introduce an erroneous factor of $45$.
Fix the candidate <statistical models>, their parameter <prior distributions>, and model <prior probabilities> before using the current data for this update. The <Bayes factor> is useful because it separates evidence from model <prior odds>, but it still depends on the parameter <prior distributions>. An <improper prior> has an arbitrary multiplicative normalization: multiplying its kernel by $c$ multiplies its formal <marginal likelihood> by $c$, even if the within-model <posterior distribution> is normalizable. Without a separately justified convention, the ordinary <Bayes factor> is then undefined up to arbitrary constants.<Bayes Factors by Robert E. Kass and Adrian E. Raftery>[Kass and Raftery, Bayes Factors (1995), sections 5.2–5.3, pp. 782–783: prior sensitivity and undefined multiplicative constants]{ref}
Back to article page
