Bayes' theorem
= Bayes' theorem
{c}
{title2=$\mathbb P(B\mid A)=\mathbb P(A\mid B)\mathbb P(B)/\mathbb P(A)$}
{wiki}
For a finite <measurable partition> $(B_j)$ with positive <probabilities>, and an <event> $A$ of positive <probability>,
$$
\mathbb P(B_j\mid A)
=\frac{\mathbb P(A\mid B_j)\mathbb P(B_j)}
{\sum_k\mathbb P(A\mid B_k)\mathbb P(B_k)}.
$$
Its beauty is that a model predicting observations from a hidden state can also describe uncertainty about the state after an observation. Each numerator is a joint <probability>; the denominator, supplied by the <law of total probability>, makes the answers sum to one. This is a direct consequence of <conditional probability>, rather than an extra axiom.
Select the coin from <law of total probability> once, then toss that same coin repeatedly, with <conditionally independent> tosses given its identity. Let $B$ denote the $3/4$-head coin, initially of <prior probability> $1/5$. After one head $H$, its <posterior probability> is
$$
\boxed{\mathbb P(B\mid H)=\frac{(1/5)(3/4)}{(1/5)(3/4)+(4/5)(1/2)}=\frac3{11}.}
$$
The <likelihood> of a head is larger under $B$, but $B$ remains less likely than the other coin because its <prior probability> was smaller. A large <likelihood> for an observation does not itself imply a large <posterior probability> for the hypothesis.
After an ordered sequence containing $h$ heads and $t$ tails, its <likelihood ratio> between the two coins is $(3/2)^h(1/2)^t$. Bayes' theorem therefore gives <posterior odds>
$$
O_{B:B^c}=\frac14\left(\frac32\right)^h\left(\frac12\right)^t,\qquad
\mathbb P(B\mid\text{sequence})=\frac{O_{B:B^c}}{1+O_{B:B^c}}.
$$
Each head multiplies the <odds> by $3/2$ and each tail by $1/2$. Three heads give <posterior probability> $27/59$, still below one half; four give $81/145$, above one half. The finite-model update and the distinction between <prior probability>, <likelihood> and <posterior probability> are set out in https://ocw.mit.edu/courses/18-05-introduction-to-probability-and-statistics-spring-2022/mit18_05_s22_statistics.pdf[MIT 18.05, Reading 11: Bayesian Updating with Discrete Priors].
\Image[/bayes-coin-updating.png]
{title=Updating uncertainty about one selected coin after repeated observations}
The hidden selection also creates dependence. Each toss has marginal head <probability> $11/20$, but two heads have joint <probability> $(4/5)(1/2)^2+(1/5)(3/4)^2=5/16$, greater than $(11/20)^2=121/400$. Their indicator <covariance> is $1/100$. Seeing a head makes the head-favoring coin more likely, so it also raises the chance of a future head. This is <covariance induced by a shared latent variable>, despite <conditional independence> of the tosses given the coin.
Historically, Thomas Bayes's essay on an unknown success <probability> appeared in 1763 after his death, communicated by Richard Price. Its opening problem asks how observed successes and failures constrain that unknown <probability>; Price's introductory letter explains the connection to reasoning from experience. That is a more substantial inference problem than simply rearranging today's elementary identity. https://www.gtfp.cs.rhul.ac.uk/pulskamp/Bayes/bayesessay-rjp.pdf[Bayes, An Essay towards solving a Problem in the Doctrine of Chances, 1763; original text transcribed, introductory letter and statement of the problem]{ref}