= Bayesian statistics
{c}
{wiki}
= Bayesian inference
{c}
{synonym}
<Bayesian statistics> expresses uncertainty about an unknown <statistical parameter> through a <probability distribution>, then uses <Bayes' theorem> to update it after observing data. The <prior distribution> describes uncertainty before the current observations; the <likelihood function> describes how the <statistical model> generates those observations at each parameter value; the <posterior distribution> describes uncertainty conditional on the observations and that model. A physical parameter can have one fixed value while its value remains uncertain to us. Assigning it a <probability distribution> represents that uncertainty under the chosen model, rather than requiring the physical parameter to change randomly between measurements.
For example, suppose ten <Bernoulli trials> are independent conditional on an unknown heads <probability> $\theta$, with a <beta distribution> <prior> $\operatorname{Beta}(2,2)$. Eight heads and two tails give the <posterior distribution> $\operatorname{Beta}(10,4)$ by <Beta-binomial conjugacy>. Its <posterior mean> is $10/14=5/7\approx0.714286$, between the <prior> <mean> $1/2$ and observed <binomial proportion> $8/10$. The <Bayesian posterior> below computes the whole update, and the <credible interval> below summarizes its remaining uncertainty. The conjugate update is explained in https://ocw.mit.edu/courses/18-05-introduction-to-probability-and-statistics-spring-2022/mit18_05_s22_class15-prep.pdf[MIT 18.05, Reading 15: beta and normal conjugate priors].
The useful feature is that one <posterior distribution> supports several questions: a <posterior mean> estimates a value, a <credible interval> reports a range, and a <posterior predictive distribution> predicts a future observation while averaging over parameter uncertainty. With conditionally independent new data, today's <posterior distribution> can become tomorrow's <prior distribution>. These conclusions remain conditional on the <statistical model> and <prior distribution>; a precise update does not establish that either assumption fits the experiment.
A historical starting point is Thomas Bayes's essay in the 1763 volume of the Royal Society's Philosophical Transactions, communicated after his death by Richard Price. Its opening problem asks for uncertainty about an unknown event <probability> given observed successes and failures: the same direction of reasoning illustrated by the coin example. See https://www.gtfp.cs.rhul.ac.uk/pulskamp/Bayes/bayesessay-rjp.pdf[Bayes and Price, An Essay towards solving a Problem in the Doctrine of Chances, opening letter and problem].
Back to article page
