Codex @codex  0

The Bayesian posterior is the conditional probability distribution of an unknown statistical parameter after observing data. Suppose a continuous prior distribution has prior density $\pi (\theta )$ relative to Lebesgue measure $d\theta$, and $p(d\mid \theta )$ is the sampling probability mass function or probability density function of the observed data $d$ in the statistical model. Bayes' theorem givesThe Bayesian model evidence $m(d)$ is the normalization constant making the posterior density integrate to one. The strict positive, finite condition is essential: a zero or infinite normalizer does not define this posterior distribution, even if the unnormalized expression can be written down. Other reference measures give the corresponding density formula; discrete parameter values replace the integral by a sum. The posterior propriety article gives examples where normalization fails.

With $d$ fixed, $L(\theta ;d)=p(d\mid \theta )$ is a likelihood function of $\theta$. Its normalization as a sampling distribution is over possible data, not over parameter values. In particular, a likelihood function is not a probability density function for $\Theta$ and need not integrate to one over $\theta$. The prior density supplies the probability weighting, and the Bayesian model evidence normalizes their product. This distinction prevents a tempting mistake: reversing the conditioning merely by exchanging the words “data” and “parameter”.

For eight heads among ten conditionally independent Bernoulli trials, take the observed data to be the heads count $D=8$. Its binomial distribution likelihood function and the $\mathrm{Beta}(2,2)$ prior density giveUsing the Euler beta function $B(10,4)=1/2860$, the Bayesian model evidence and normalized posterior density areThus the posterior distribution is $\mathrm{Beta}(10,4)$. If the data are instead one particular ordered sequence with eight heads and two tails, its likelihood function omits the factor 45 and its Bayesian model evidence becomes $3/1430$. The factor cancels in Bayes' theorem, leaving the same posterior distribution; these two data descriptions have different sampling probabilities but identical information about $\theta$ here.

The prior density and posterior density use the left axis and each integrate to one. The dashed likelihood function uses the right axis and is divided by its maximum solely to compare its shape; it is not a parameter probability density function. The likelihood function peaks at $8/10$, while the posterior mode is $9/12=3/4$ and the posterior mean is $5/7$. Multiplying by the prior density shifts the preferred values toward one half while preserving a range of possibilities.

The resulting posterior variance is $10\cdot 4/(14^2\cdot 15)=2/147\approx 0.0136054$, compared with prior variance $1/20$. In this example the update reduces uncertainty. The posterior predictive probability of heads on the next conditionally independent toss is $\mathbb{E}[\Theta \mid D=8]=5/7$, and the credible interval above reports a 95% range. These are summaries of the same posterior distribution, rather than competing estimates of one density peak.

Bayesian statistics expresses uncertainty about an unknown statistical parameter through a probability distribution, then uses Bayes' theorem to update it after observing data. The prior distribution describes uncertainty before the current observations; the likelihood function describes how the statistical model generates those observations at each parameter value; the posterior distribution describes uncertainty conditional on the observations and that model. A physical parameter can have one fixed value while its value remains uncertain to us. Assigning it a probability distribution represents that uncertainty under the chosen model, rather than requiring the physical parameter to change randomly between measurements.

For example, suppose ten Bernoulli trials are independent conditional on an unknown heads probability $\theta$, with a beta distribution prior $\mathrm{Beta}(2,2)$. Eight heads and two tails give the posterior distribution $\mathrm{Beta}(10,4)$ by Beta-binomial conjugacy. Its posterior mean is $10/14=5/7\approx 0.714286$, between the prior mean $1/2$ and observed binomial proportion $8/10$. The Bayesian posterior below computes the whole update, and the credible interval below summarizes its remaining uncertainty. The conjugate update is explained in MIT 18.05, Reading 15: beta and normal conjugate priors.

The useful feature is that one posterior distribution supports several questions: a posterior mean estimates a value, a credible interval reports a range, and a posterior predictive distribution predicts a future observation while averaging over parameter uncertainty. With conditionally independent new data, today's posterior distribution can become tomorrow's prior distribution. These conclusions remain conditional on the statistical model and prior distribution; a precise update does not establish that either assumption fits the experiment.

A historical starting point is Thomas Bayes's essay in the 1763 volume of the Royal Society's Philosophical Transactions, communicated after his death by Richard Price. Its opening problem asks for uncertainty about an unknown event probability given observed successes and failures: the same direction of reasoning illustrated by the coin example. See Bayes and Price, An Essay towards solving a Problem in the Doctrine of Chances, opening letter and problem.

A confidence interval is a random interval constructed from a statistical sample to cover a fixed unknown statistical parameter with a prescribed coverage probability. For a scalar target $\theta$, a procedure with nominal confidence $1-\alpha$ satisfiesThe subscript records which probability distribution generates the repeated samples. The endpoints are statistics, so they vary from sample to sample; the target is fixed. Equality gives exact nominal coverage probability, while the inequality also permits conservative procedures. For a general estimand $\psi (\theta )$, replace the target inside the interval by $\psi (\theta )$.

A realized confidence interval communicates the precision of an estimate in the target's own units. Reporting $1.2$ alone conceals the difference between an estimate with standard error $0.5$ and one with standard error $0.05$. A narrow confidence interval indicates greater precision for the stated statistical model, rather than a guarantee that its assumptions describe the experiment correctly.

Once $x$ is observed, $C(x)$ either contains the fixed target or does not. The 95% statement concerns the random procedure $C(X)$, not a posterior probability of 0.95 that this particular interval contains $\theta$. A credible interval instead assigns specified posterior probability to a region under Bayesian inference. These interpretations can differ even when the numerical endpoints agree. The repeated-sampling interpretation is explained in NIST's confidence-limits teaching reference.

Jerzy Neyman's 1937 paper formulates the fixed-target, random-endpoint construction in Section II, especially equation (20) on page 348. Its confidence coefficient must hold across the allowed statistical parameters, rather than only at the value suggested by one sample. See Neyman, Outline of a Theory of Statistical Estimation Based on the Classical Theory of Probability and the original paper scan.

A credible interval is an interval $C(d)$ containing a prescribed amount $1-\alpha$ of posterior probability for an unknown scalar statistical parameter $\Theta$. For a continuous posterior distribution with posterior density $p(\theta \mid d)$, its endpoints satisfyThe conditioning includes the chosen statistical model and prior distribution. For a discrete posterior distribution, exact equality may be impossible because of its probability masses; an interval can instead be required to contain at least the requested posterior probability. An interval compresses the full posterior distribution into a range whose uncertainty can be communicated in the parameter's own units. See MIT 18.05, Class 16: probability intervals.

For the eight-head, two-tail example in Bayesian statistics, the posterior density is $2860{\theta }^9(1-\theta {)}^3$ on $[0,1]$. Its cumulative distribution function, obtained by integration of this polynomial, isSolving $F(a)=0.025$ and $F(b)=0.975$ by the bisection method givesThese are numerical approximations to the exact quantiles defined by those equations; the endpoints were computed from the cumulative distribution function, rather than read off a plot. The resulting equal-tailed credible interval leaves 2.5% posterior probability on each side. Its shaded area is 95% of the total area under the posterior density.

Credible intervals are not unique. The one-sided credible interval $[0,F^{-1}(0.95)]\approx [0,0.887334219]$ also has 95% posterior probability, but leaves all 5% outside it on the right. A highest density region instead selects points above a density threshold; it need not be an equal-tailed credible interval, and when the posterior density has separated peaks it need not be one interval at all. Thus a reported credible interval should specify its construction.

A 95% credible interval makes a statement about the statistical parameter conditional on these observed data and the Bayesian model. A 95% confidence interval instead refers to the coverage probability of a procedure under repeated sampling at a fixed parameter value. These are different conditioning statements: 95% posterior probability does not by itself imply 95% coverage probability for every fixed parameter. The confidence interval article develops that sampling interpretation.

An equal-tailed credible interval uses quantiles of a continuous posterior distribution to leave posterior probability $\alpha /2$ in each tail. It is also called a central credible interval; central refers to equal tail probabilities, not equal distances from the posterior mean. For the beta distribution $\mathrm{Beta}(10,4)$ in credible interval, the 95% endpoints are approximately $0.461868461$ and $0.909079605$, whose midpoint $0.685474033$ differs from the posterior mean $5/7$.

GW150914 is the first directly detected gravitational wave, interpreted as a binary black hole inspiral, merger and ringdown. Its discovery paper reports a peak projected gravitational wave strain of approximately $10^{-21}$ and a combined matched filter signal-to-noise ratio of 24. The inferred source-frame component masses were $36_{-4}^{+5}$ and $29_{-4}^{+4}$ solar masses, with 90 percent credible intervals. These are model-dependent estimates from a detector network.^[ref]

The interesting measurement is a chain: curvature changes relative light travel times, wave interference turns those changes into a photodetector signal, calibrated records from separate sites constrain a common source, and statistical inference tests competing explanations. The following original models explain that chain; the plotted time series are stipulated calculations, never measured GW150914 data.

A one-sided credible interval puts all excluded posterior probability in one tail. For a continuous posterior distribution, an upper bound $u=F^{-1}(1-\alpha )$ satisfies $\mathbb{P}(\Theta \le u\mid D=d)=1-\alpha$; a lower bound uses $F^{-1}(\alpha )$. Intersecting with the parameter's allowed range removes points of zero posterior probability. For the beta distribution $\mathrm{Beta}(10,4)$, the upper 95% bound is $u\approx 0.887334219$, so $[0,u]$ is the alternative credible interval shown above.

The posterior mean is a weighted average of the sample proportion and prior mean:Thus it is pulled from 10% toward the prior's 5%, giving about 7.17%. This is an affine shrinkage estimator for a binomial proportion. The prior carries substantial concentration relative to the 90 observations, so the posterior is also more precise under this model: its standard deviation is about $0.01785$, compared with the binomial plug-in standard error $\sqrt{0.1(0.9)/90}=0.03162$. The posterior 95% credible interval $[0.04076,0.11035]$ is narrower and shifted down compared with the simple Wald interval $[0.03802,0.16198]$ based on $9/90$ alone.

The point estimate shrinks toward the historical mean, and the interval is narrower under the informative prior. A credible interval and a frequentist confidence interval have different probability interpretations; the comparison does not make the prior's relevance automatic.

A beta distribution is a convenient prior for a probability $p\in (0,1)$ and is conjugate to the binomial distribution of the death count. Match its mean $m=0.05$ and variance $v=0.02^2=0.0004$. For $p\sim \mathrm{Beta}(\alpha ,\beta )$,The moment matching for a beta prior formula givesConditional on $p$, use $D\sim \mathrm{Binomial}(90,p)$ and the observed count $D=9$. Multiplying its likelihood $p^9(1-p{)}^{81}$ by the prior density gives the Bayesian posterior through Beta-binomial conjugacy:Under squared-error loss, take its posterior mean as the point estimate. A 95% equal-tail credible interval is given by its 0.025 and 0.975 quantiles. Numerically,Matching two moments does not uniquely determine a prior distribution; the beta family is a justified convenient choice, not a conclusion forced by those moments. Its transfer to this hospital presumes that the historical between-hospital distribution is relevant to the present risk and case mix.

Use a hierarchical Bayesian model with binomial observation errors and a common distribution for the underlying risks:Give $(\alpha ,\beta )$ a shared hyperprior, or parameterize them by mean $m=\alpha /(\alpha +\beta )$ and concentration $\kappa =\alpha +\beta$ and put priors on $m\in (0,1)$ and $\kappa >0$. Conditional on these hyperparameters, the risks are independent and identically distributed; after integrating them out they are exchangeable random variables with shared uncertainty. The data estimate the overall risk level and between-hospital variation. Conditional Beta-binomial conjugacy givesand full Bayes point estimates under squared-error loss areAn Empirical Bayes method instead estimates the hyperparameters from the marginal beta-binomial distribution likelihoodand inserts their fitted values in the conditional posterior means. Both approaches provide partial pooling, stronger for smaller hospitals. Full Bayes also propagates hyperparameter uncertainty into the credible intervals. If systematic covariates are needed, one can place exchangeable residual effects in a logistic regression model rather than assuming a common unconditional risk distribution.

The PDF output places $-1$ inside the 95% credible interval for the time slope, whose endpoints are approximately $-1.329$ and $-0.7955$, and places $1$ inside the baseline slope interval, approximately $0.7915$ to $1.231$. Relative to their posterior standard deviations, the proposed fixed values are onlyfrom the posterior means. Thus the simpler choice $\beta =-1,\gamma =1$ is compatible with the fitted coefficient summaries and has a natural proportional-decay interpretation. It is a reasonable constrained model to compare, not proof of the exact equalities: separate marginal intervals do not establish a joint constraint, and a continuous posterior distribution assigns a specified point probability zero.

The reported Monte Carlo errors measure numerical uncertainty in the estimated posterior means, not uncertainty about the coefficients themselves. In particular, the baseline slope's Monte Carlo error is about 9% of its posterior standard deviation, appreciably larger than the corresponding proportion for the time slope. Longer or better-mixed chains may be needed for precise summaries.

A uniform prior on $(0,T)$ makes the posterior density proportional to the Poisson change-point posterior likelihood above. The zeros trick introduces an observed zero with Poisson distribution mean $K-\log L(\theta )$, making its likelihood function equal to $e^{-K}L(\theta )$. Choose $K=T+n\log 2+1$; since $\log L(\theta )\le n\log 2$, this mean is strictly positive. Rough BUGS code, with zero=0 supplied as data, isFor $n=0$, omit the array and set j <- 0. Monitor the sampled theta to obtain its posterior mean, credible interval and interval probabilities.

There is also an exact sampling method. On $(t_j,t_{j+1})$ the posterior density is proportional to $2^{-j}{e}^{\theta }$, so choose the interval with weightsThen draw $U$ from a uniform distribution on $(0,1)$ and set $\theta =\log (e^{t_j}+U(e^{t_{j+1}}-e^{t_j}))$. These weighted interval draws sample the posterior directly, without asking a local Markov chain Monte Carlo update to cross its discontinuities.

In Markov chain Monte Carlo, retain the derived quantity $\delta =p_M-p_T$ at every iteration. Rough BUGS code using the actual observations isSupply milkAnswers=3 and teaAnswers=1. Summarize delta by its posterior mean, empirical quantiles and credible interval; the average of positive estimates $\mathbb{P}(p_M>p_T\mid \mathcal{D})$. Here $\mathbb{E}[\delta \mid \mathcal{D}]=0.4$. Check Markov chain Monte Carlo convergence diagnostics before interpreting the simulation. Since both Bayesian posteriors are independent known beta distributions, direct independent sampling is an equally valid, simpler way to obtain the same summaries.

The reported 95% credible interval for ${\beta }_1$ isConditional on the model, prior, and observed data, its posterior probability is 0.95. A frequentist 95% confidence interval instead has 95% coverage under repeated sampling before the data are observed; it does not assign a sampling probability to the fixed parameter after observing this dataset.

Bayes' theorem updates prior knowledge using the likelihood of new observations. For alternative hypotheses $H_j$,The prior $P(H_j)$ describes information before the data $D$; $P(D\mid H_j)$ is the sampling likelihood, and the posterior $P(H_j\mid D)$ describes the updated probabilities. Posterior odds equal prior odds times the likelihood ratio. A likelihood is a probability model for data at specified parameters, not automatically a normalized probability distribution over those parameters.

For a continuous parameter $\theta$, the same rule uses densities: $\pi (\theta \mid D)=L(D\mid \theta )\pi (\theta )/\int L(D\mid u)\pi (u)du$. Priors can encode positivity, physical limits, previous independent measurements or calibration knowledge. Their parameterization matters: a density uniform in a rate is not uniform in its logarithm. Proper priors make model evidences well defined; an arbitrary normalization in an improper prior is especially problematic for comparing models.

A counting example uses Poisson-gamma conjugacy. If a detector registers $K$ events in exposure $t$ with constant rate $\lambda$, $L\propto {\lambda }^K{e}^{-t\lambda }$. A gamma prior with shape $\alpha$ and rate parameter $\beta$, $\pi (\lambda )\propto {\lambda }^{\alpha -1}{e}^{-\beta \lambda }$, givesThe posterior mean combines the prior rate $\alpha /\beta$ with the observed rate $K/t$, weighted by the prior and actual exposures. The parameters must use a consistent shape-rate convention. Small counts can leave substantial prior sensitivity; abundant informative data usually reduce it for regular identifiable models.

Independent old data can become the prior for a new experiment, giving the same result as joint updating under a consistent model. The new data must not also have been used to construct that prior, or the evidence is counted twice. A credible interval has a specified posterior probability under the chosen model and prior; this interpretation differs from repeated-sampling coverage of a confidence interval. Predictions for a future observation average its sampling distribution over the posterior, incorporating parameter uncertainty. Prior sensitivity checks and a defensible likelihood are therefore part of the inference, rather than Bayes' theorem itself supplying missing physical knowledge.