Bayesian statistics Created 2026-09-24 Updated 2026-10-14
Bayesian statistics expresses uncertainty about an unknown statistical parameter through a probability distribution, then uses Bayes' theorem to update it after observing data. The prior distribution describes uncertainty before the current observations; the likelihood function describes how the statistical model generates those observations at each parameter value; the posterior distribution describes uncertainty conditional on the observations and that model. A physical parameter can have one fixed value while its value remains uncertain to us. Assigning it a probability distribution represents that uncertainty under the chosen model, rather than requiring the physical parameter to change randomly between measurements.
For example, suppose ten Bernoulli trials are independent conditional on an unknown heads probability , with a beta distribution prior . Eight heads and two tails give the posterior distribution by Beta-binomial conjugacy. Its posterior mean is , between the prior mean and observed binomial proportion . The Bayesian posterior below computes the whole update, and the credible interval below summarizes its remaining uncertainty. The conjugate update is explained in MIT 18.05, Reading 15: beta and normal conjugate priors.
The useful feature is that one posterior distribution supports several questions: a posterior mean estimates a value, a credible interval reports a range, and a posterior predictive distribution predicts a future observation while averaging over parameter uncertainty. With conditionally independent new data, today's posterior distribution can become tomorrow's prior distribution. These conclusions remain conditional on the statistical model and prior distribution; a precise update does not establish that either assumption fits the experiment.
A historical starting point is Thomas Bayes's essay in the 1763 volume of the Royal Society's Philosophical Transactions, communicated after his death by Richard Price. Its opening problem asks for uncertainty about an unknown event probability given observed successes and failures: the same direction of reasoning illustrated by the coin example. See Bayes and Price, An Essay towards solving a Problem in the Doctrine of Chances, opening letter and problem.
The displayed pointwise control limits follow from the normal approximation to an observed binomial proportion. Clip plotted limits to the probability range. Their nominal coverage is approximate and can be poor for small expected counts; exact binomial tails give a discrete alternative. Pointwise coverage does not provide simultaneous coverage of all hospitals; use Bonferroni correction or another familywise error rate procedure when that is the required guarantee.
Moment matching for a beta prior turns a desired mean and variance for an unknown probability into the parameters of a beta distribution prior. Choose and
Solving the beta distribution mean and variance identities gives
The strict upper bound on the variance makes , so both parameters are positive. Substitution gives mean and variance exactly. A smaller specified variance at fixed mean produces a more concentrated prior distribution.
For example, a desired prior mean and variance give , hence the beta distribution prior . With the same mean and variance , the calculation instead gives and the broader beta distribution prior . Both appear in the concentration diagram above.
The excluded boundary cases have clear meanings. Zero variance would require with certainty, not a beta distribution with finite positive parameters. At the other boundary, any random variable satisfies , hence variance at most ; equality requires to take only the endpoint values zero and one. This is a Bernoulli distribution, not a beta distribution. Thus cannot be matched by positive finite parameters, and or is also excluded.
After observing Bernoulli trials, the Beta-binomial conjugacy makes the posterior mean a weighted average of the prior mean and observed binomial proportion :
This formula gives a precise interpretation as the prior's weight relative to the sample size in this posterior mean. It does not require fictitious observed counts: the beta distribution parameters can be any positive real numbers. For the eight-head, two-tail example above, and , so the posterior mean is .
// Target: statistical-inference.bigb
Four distinct objections, each with a corresponding improvement, are as follows.
  • Different patients imply different baseline risks. Raw death rates confound performance with case mix, including urgency, disease severity and comorbidity. Use a prespecified, validated risk-adjusted provider comparison, standardizing to comparable patients or comparing observed deaths with . Check overlap and model calibration rather than assuming adjustment removes all confounding.
  • Small annual volumes produce unstable rankings. A few deaths can move a surgeon far up or down a league table; zero deaths need not indicate zero risk. Display counts and suitably calibrated confidence intervals for binomial proportions, preferably in funnel plots against volume. Pool suitable periods or use a hierarchical Bayesian model to stabilize estimates, while allowing genuine time changes.
  • Many simultaneous comparisons generate chance outliers. With independent null comparisons, the probability of at least one false positive is , and ranks amplify selection of extremes. Use multiple testing adjustments or simultaneous control limits and seek confirmation before interpreting an apparent outlier. Dependence through the national comparator must also be accounted for.
  • The endpoint and unit of attribution can mislead. In-hospital death depends on discharge and transfer policy, and care is delivered by a team; follow-up after discharge is omitted. Use an independently audited fixed-horizon endpoint, such as a prespecified 30-day mortality outcome, linked across hospitals and to death registrations, and report an appropriate team or institutional comparison alongside any surgeon-level estimate. Reliable attribution and consistent inclusion criteria reduce incentives to manipulate coding or case selection.
A confidence interval containing the national rate does not demonstrate equivalence, and a raw ranking does not establish a causal performance difference. The improved presentation should convey precision and uncertainty as well as the estimated outcome.
Use a conventional two-sided test at significance level . Let per arm, , and . Under the specified alternative, the normal approximation to the difference of independent binomial proportions has standard deviation
For planning the pooled null standard error, use , giving . The rejection threshold is approximately , exceeding the mean alternative difference. With the standard normal distribution function, the approximate statistical power is
No: the usual two-sided 5% calculation gives about 45% power, below 50%. The expected death counts are only 20 and 10, so an exact discrete design calculation could refine this approximation.
The question does not specify the tail convention. If a beneficial direction is prespecified and a one-sided test at 5% is justified, replace by approximately ; the resulting statistical power is about , and the answer is then yes. This change of testing convention must be declared.
If , the observed binomial proportion has variance . Consequently its standard error is proportional to . A common target therefore produces broad uncertainty at low volume and narrower uncertainty at high volume, forming the funnel-shaped envelope around the horizontal target. The narrowing represents sampling precision, not a predicted change of mortality with volume.