Bayesian posterior ($p(\theta \mid d)=p(d\mid \theta )\pi (\theta )/m(d)$)

The Bayesian posterior is the conditional probability distribution of an unknown statistical parameter after observing data. Suppose a continuous prior distribution has prior density $\pi (\theta )$ relative to Lebesgue measure $d\theta$, and $p(d\mid \theta )$ is the sampling probability mass function or probability density function of the observed data $d$ in the statistical model. Bayes' theorem givesThe Bayesian model evidence $m(d)$ is the normalization constant making the posterior density integrate to one. The strict positive, finite condition is essential: a zero or infinite normalizer does not define this posterior distribution, even if the unnormalized expression can be written down. Other reference measures give the corresponding density formula; discrete parameter values replace the integral by a sum. The posterior propriety article gives examples where normalization fails.

With $d$ fixed, $L(\theta ;d)=p(d\mid \theta )$ is a likelihood function of $\theta$. Its normalization as a sampling distribution is over possible data, not over parameter values. In particular, a likelihood function is not a probability density function for $\Theta$ and need not integrate to one over $\theta$. The prior density supplies the probability weighting, and the Bayesian model evidence normalizes their product. This distinction prevents a tempting mistake: reversing the conditioning merely by exchanging the words “data” and “parameter”.

For eight heads among ten conditionally independent Bernoulli trials, take the observed data to be the heads count $D=8$. Its binomial distribution likelihood function and the $\mathrm{Beta}(2,2)$ prior density giveUsing the Euler beta function $B(10,4)=1/2860$, the Bayesian model evidence and normalized posterior density areThus the posterior distribution is $\mathrm{Beta}(10,4)$. If the data are instead one particular ordered sequence with eight heads and two tails, its likelihood function omits the factor 45 and its Bayesian model evidence becomes $3/1430$. The factor cancels in Bayes' theorem, leaving the same posterior distribution; these two data descriptions have different sampling probabilities but identical information about $\theta$ here.

The prior density and posterior density use the left axis and each integrate to one. The dashed likelihood function uses the right axis and is divided by its maximum solely to compare its shape; it is not a parameter probability density function. The likelihood function peaks at $8/10$, while the posterior mode is $9/12=3/4$ and the posterior mean is $5/7$. Multiplying by the prior density shifts the preferred values toward one half while preserving a range of possibilities.

The resulting posterior variance is $10\cdot 4/(14^2\cdot 15)=2/147\approx 0.0136054$, compared with prior variance $1/20$. In this example the update reduces uncertainty. The posterior predictive probability of heads on the next conditionally independent toss is $\mathbb{E}[\Theta \mid D=8]=5/7$, and the credible interval above reports a 95% range. These are summaries of the same posterior distribution, rather than competing estimates of one density peak.

For independent and identically distributed observations from a normal distribution with known variance ${\sigma }^2$, an exponential distribution prior of rate $\lambda$ on their nonnegative mean gives this truncated normal distribution posterior. Complete the square in $-n(\mu -\bar{x}{)}^2/(2{\sigma }^2)-\lambda \mu$; the truncation remains essential even when the untruncated normal mean is negative.

A posterior moment is a moment computed under a posterior distribution. Its first raw moment is the posterior mean; its second central moment is the posterior variance.

For one observation from a continuous uniform distribution on $[0,\theta ]$ and prior density $\theta e^{-\theta }$, the likelihood cancels the factor $\theta$ and leaves $e^{-\theta }{1}_{\{\theta \ge x\}}$. Normalization gives posterior density $e^{-(\theta -x)}{1}_{\{\theta \ge x\}}$, for $x\ge 0$. Its mean is $x+1$ and variance one. The Bayes estimator under squared error loss is therefore $x+1$, unchanged by multiplying the loss by a positive constant.

For independent observations uniform on $(\theta -d,\theta +d)$ and a uniform prior on $(A,B)$, the likelihood is constant wherever all data-compatible intervals overlap. The posterior is therefore uniform on the displayed intersection, provided it has positive length. Both squared-error and absolute-error Bayes estimates are its midpoint. An empty or zero-length interval has zero marginal likelihood and cannot be normalized into this ordinary posterior density.

For one uniformly sampled integer $Y\in \{1,\dots ,N\}$, the likelihood function for the unknown endpoint is $N^{-1}{1}_{N\ge y}$. A flat improper prior on positive integers gives an improper posterior distribution, since the harmonic series diverges. A reciprocal prior instead gives a proper posterior proportional to $N^{-2}$ for $N\ge y$, with approximate survival probability $y/n$ and finite quantiles, but infinite posterior mean. A uniform prior cut off at $M$ gives posterior mean $(M-y+1)/(H_M-H_{y-1})\sim M/\log M$, demonstrating sensitivity to that arbitrary cutoff.

For continuously distributed uncertain quantities, define rank one to be the largest. A joint posterior distribution induces a discrete distribution on each quantity's rank. Estimate it by ranking the whole vector in each joint draw and averaging rank indicators; the probability of rank one is the probability of being largest. Ranking posterior means does not recover this uncertainty. The joint draws must preserve any posterior dependence, and tie conventions must be specified for models with atoms.

A posterior variance is the variance of a parameter under its Bayesian posterior, namely $\mathbb{E}[(\theta -\mathbb{E}[\theta \mid y]{)}^2\mid y]$ when finite. It describes remaining uncertainty after observing data and is distinct from the repeated-sampling variance of a fitted estimator.

For a vector parameter with finite second moments and posterior mean $m_y$, the posterior covariance matrix records its uncertainty and linear dependence after conditioning on the observations. Its diagonal entries are posterior variances. In a regular locally flat-prior normal distribution approximation it is the inverse observed information. A quadratic form has posterior expectation equal to the form at the posterior mean plus the trace of its matrix times this covariance.

The distribution of a new or missing quantity integrates its model over the Bayesian posterior: $p(y^{\ast }\mid y)=\int p(y^{\ast }\mid \varphi ,y)p(\varphi \mid y)d\varphi$. Both parameter and residual uncertainty should enter predictive draws used for multiple imputation.

If future Bernoulli random variables are conditionally independent given a shared parameter $\Theta$, with success probability $\pi (\Theta )$, their all-success conditional probability is $\pi (\Theta {)}^k$. Taking a posterior expectation proves the identity. For $k\ge 1$, Jensen's inequality and $0\le \pi \le 1$ give $(\mathbb{E}[\pi \mid D]{)}^k\le \mathbb{E}[{\pi }^k\mid D]\le \mathbb{E}[\pi \mid D]$. For two distinct future trials the posterior covariance is $\mathbb{E}[{\pi }^2\mid D]-(\mathbb{E}[\pi \mid D]{)}^2=\mathrm{Var}(\pi \mid D)$. Thus shared parameter uncertainty can induce positive dependence even when trials are conditionally independent.

A posterior predictive check compares a summary of the observed data with the same summary in replicated data drawn from a model's posterior predictive distribution. The replication must match the target: replicating an existing group's observations conditional on its effect differs from predicting a new group with a newly drawn effect. Such checks expose mismatches in dispersion, tails or other features; using the observations to fit and check the model means they are not automatically calibrated frequentist tests.

In a hierarchical Bayesian model, replication conditional on fitted observation-level random effects checks the sampling layer. Replication with fresh random effects drawn from their population law also checks the population mean and variation. To assess a dose-response curve, sample the global parameters from their posterior distribution, draw fresh effects for new plates at each existing dose, and compare replicated means, spreads and trend discrepancies with the observed values. Reusing the fitted plate effects can conceal a misspecified mean curve by reproducing deviations those effects already absorbed.

For independent observations $y_s\sim N(\beta +d_s,V_s)$, put $a_s=V_s^{-1}$, $S={\sum }_s{a}_s$, $r_s=y_s-d_s$, $\bar{r}=S^{-1}{\sum }_s{a}_s{r}_s$ and $Q={\sum }_s{a}_s(r_s-\bar{r}{)}^2$. Integrating the likelihood function against a flat improper prior on $\beta$ gives, up to the arbitrary prior constant,Indeed ${\sum }_s{a}_s(r_s-\beta {)}^2=Q+S(\beta -\bar{r}{)}^2$, and the remaining one-dimensional Gaussian integral is $\sqrt{2\pi /S}$. The conditional Bayesian posterior of $\beta$ is $N(\bar{r},S^{-1})$.

A posterior predictive probability averages a future-event probability over the Bayesian posterior. For a Bernoulli distribution with unknown parameter $p$, it is $\mathbb{E}[p\mid \mathcal{D}]$. Under Beta-binomial conjugacy with posterior $\mathrm{Beta}(a,b)$, the predictive success probability is $a/(a+b)$.

A posterior density is the density of a posterior distribution with respect to a chosen reference measure.

The log-posterior is the logarithm of a positive posterior density. Its gradient is the posterior score control variate when boundary terms vanish. An additive constant independent of the parameter does not affect optimization or differentiation.

The posterior mean is the expected value of a parameter under its posterior distribution; under squared-error loss it is the Bayes estimator.

Write the log posterior kernel as $H_n(\theta )=\log p(\theta )+{\sum }_i\log f(Y_i\mid \theta )$, with a unique concentrating interior mode $\tilde{\theta }$ and $J_n=-H_n^{\prime \prime }(\tilde{\theta })>0$. Apply Laplace's method to the numerator and denominator of the expected value under the posterior distribution. Their common leading Gaussian factor cancels. Under smoothness and localization, the first correction is $g^{\prime \prime }(\tilde{\theta })/(2J_n)+g^{\prime }(\tilde{\theta })H_n^{\prime \prime \prime }(\tilde{\theta })/(2J_n^2)$, of order $n^{-1}$ when the posterior curvature and log-kernel derivatives have their usual order $n$.

A posterior probability is the probability assigned to an event by the posterior distribution after conditioning on the observed data.

For two exhaustive mutually exclusive models, posterior odds are the ratio of their posterior probabilities. Bayes' theorem givesThe second factor is the Bayes factor, formed from model prior predictive distributions. Evidence favouring one model need not make that model's posterior probability exceed one half if its prior probability was sufficiently small.

When the prior odds of a real effect are $R$, a study's Type I error is $\alpha$, and its statistical power is $1-\beta$, a positive report updates those odds to $R(1-\beta )/\alpha$. This follows directly from Bayes' theorem. A small false-positive rate does not itself make the effect's posterior probability high: the prior odds and power also matter. The posterior exceeds one half exactly when $R>\alpha /(1-\beta )$.

A point-null mixture prior assigns positive mass to one parameter value and distributes the remaining mass continuously over alternatives. Posterior point mass is obtained by dividing the null component of the marginal density by the full marginal density.

For $\overline{X}\mid \mu \sim N(\mu ,1/n)$ and an equal mixture of $\mu =0$ and $\mu \sim N(0,{\tau }^2)$,

For the equal point-null mixture,

With a fixed diffuse alternative prior, a frequentist p-value and the posterior probability of a point null can order evidence differently. For $n=100$ and $\tau =1$, the p-value exceeds the null posterior near zero, while sufficiently far in the tails the null posterior exceeds the p-value because it has a slightly slower Gaussian decay.

An improper prior is a nonnegative prior kernel with infinite total mass. It can still produce a proper posterior after multiplication by the likelihood and normalization.

The Haldane prior has formal beta distribution parameters $(0,0)$. It is an improper prior, with infinite mass at both endpoints of $(0,1)$. For a binomial distribution likelihood with $s$ successes and $f$ failures, its posterior is $\mathrm{Beta}(s,f)$ only when $s,f>0$. A density flat in the log odds induces this kernel by the Jacobian determinant $d\log [p/(1-p)]/dp=1/[p(1-p)]$.

The measure $d\sigma /\sigma$ is invariant under multiplication by every positive constant. In the logarithmic coordinate it is Lebesgue measure. It cannot be normalized over $(0,\infty )$, although a bounded truncation is a log-uniform distribution.

A random-effect model with positive marginal likelihood at scale zero cannot yield a proper posterior distribution under a prior proportional to the reciprocal scale near zero. On a compact set of the other parameters, continuity and positivity give a positive likelihood lower bound, while ${\int }_0^{\epsilon }d\lambda /\lambda =\infty$. Individual full conditional distributions can still be proper, so their existence alone does not justify a Gibbs sampler with a probability distribution as its joint target.

A posterior kernel $L(\theta )\pi (\theta )$ defines a posterior distribution exactly when its integral is finite and nonzero. Proper full conditional densities do not suffice. For example, if a known positive measurement variance makes a marginal normal likelihood approach a strictly positive limit as a latent variance $v$ tends to zero, the log-flat prior $dv/v$ gives infinite posterior mass at that boundary. The same problem occurs for a latent exponential scale $\tau$ with $d\tau /\tau$ when the observed likelihood has a positive limit as the exponential contribution vanishes.

If a model configuration leaves a parameter absent from its likelihood function, giving that parameter an independent flat improper prior makes the posterior distribution normalization infinite whenever the configuration has positive prior distribution mass. Integrate first over that parameter: a positive constant likelihood function factor is repeated over all of $\mathbb{R}$. This occurs for the second segment mean of a Gaussian change-point detection model when the proposed split is after the final observation. There is no posterior distribution or valid Gibbs sampler for that joint kernel. A proper prior distribution on the unused parameter is necessary if the no-change configuration is retained.

For conditionally independent Poisson observations of mean $\theta$ and a gamma distribution prior of shape $r>0$ and rate $\lambda >0$, the likelihood function contributes ${\theta }^{{\sum }_i{x}_i}{e}^{-n\theta }$. Multiplying by the prior density gives a posterior distribution with shape $r+{\sum }_i{x}_i$ and rate $\lambda +n$. The single-observation update is shape $r+X$ and rate $\lambda +1$.

The posterior mean is the convex combinationThus the prior mean $r/\lambda$ and the sample maximum-likelihood estimate $\bar{x}$ bracket it. Integrating the next conditional Poisson distribution against this posterior distribution gives the Poisson-gamma mixture, a negative binomial distribution; its spread includes uncertainty in $\theta$ as well as the conditional count variation.

If the positive baseline rate has a gamma distribution of shape $a$ and rate $b$, its logarithm has this probability density function with respect to $d\mu$. The extra factor $e^{\mu }$ is the Jacobian determinant of the transformation. After a Gibbs sampler draws the conditional baseline rate, taking its logarithm is an exact update of the intercept. Omitting the Jacobian changes the prior.

Given $\Theta$, independent observations $Y_j$ have Poisson distribution with means $m_j\Theta$ for known exposures. A gamma distribution prior of shape $\alpha$ and rate $\beta$ yields a gamma posterior with shape $\alpha +{\sum }_j{Y}_j$ and rate $\beta +{\sum }_j{m}_j$. The likelihood kernel is ${\theta }^{{\sum }_j{Y}_j}{e}^{-\theta {\sum }_j{m}_j}$. Thus the posterior mean depends on total claims and total exposure, not the number of periods alone.

A gamma prior with shape $\alpha$ and rate $\beta$ combined with $n$ independent observations from an exponential distribution of rate $\theta$ gives the posteriorUnder squared-error loss, the Bayes estimator is its posterior mean $(\alpha +n)/(\beta +{\sum }_i{X}_i)$.

A beta prior $\mathrm{Beta}(\alpha ,\beta )$ combined with $s$ Bernoulli successes and $f$ failures gives posterior $\mathrm{Beta}(\alpha +s,\beta +f)$.

With a uniform prior distribution on a Bernoulli chance $\theta$, $n$ successes yield posterior density $(n+1){\theta }^n$. Integration gives next-trial success probability $(n+1)/(n+2)$, but the singleton $\theta =1$ retains probability zero. More generally, ordinary conditioning with a finite integrable likelihood and positive finite normalization constant preserves absolute continuity, so a continuous prior does not acquire a new atom at certainty from a finite sample.

For independent geometric random variables with success probability $p$, observing $d$ terminal events and $F$ preceding failed Bernoulli trials gives likelihood function proportional to $p^d(1-p{)}^F$. Multiplication by a beta distribution prior $\mathrm{Beta}(a,b)$ gives the posterior distribution $\mathrm{Beta}(a+d,b+F)$. With complete waiting times $x_1,\dots ,x_n$, take $d=n$ and $F={\sum }_i(x_i-1)$.

A waiting time that is right-censored after $c$ failed trials contributes $(1-p{)}^c$: increase $F$ by $c$ without increasing $d$. This requires the censoring mechanism to be ignorable, as with an externally fixed observation horizon. Counting a censored waiting time as an observed event changes the likelihood function and biases the update.

Mix a binomial distribution with a beta distribution for its success probability: $\Theta \sim \mathrm{Beta}(\alpha ,\beta )$ and $Y\mid \Theta \sim \mathrm{Binomial}(n,\Theta )$. Integrating their densities gives the displayed mass for $y=0,\dots ,n$. Its expectation is $n\alpha /(\alpha +\beta )$, and its variance is $n\alpha \beta (\alpha +\beta +n)/[(\alpha +\beta {)}^2(\alpha +\beta +1)]$. It is the posterior predictive distribution for repeated Bernoulli trials after a beta-prior update.

A uniform prior distribution on the success chance makes the number of successes in any fixed number $n$ of Bernoulli trials uniform on $0,\dots ,n$, because $\left(\genfrac{}{}{0ex}{}{n}{y}\right)\mathrm{B}(y+1,n-y+1)=1/(n+1)$. This is a statement about counts, not a uniform distribution over the $2^n$ ordered outcome sequences. It also makes predictions reversible between equally sized batches: their joint probability distribution is symmetric and their marginal count distributions are the same uniform distribution.

Conditional on the pooled number $s$ of successes in $m+n$ exchangeable random variables that are binary, all placements of those successes are equally likely under a common success-chance mixture. The count in a chosen batch of size $n$ therefore has a hypergeometric distribution, with mass $\left(\genfrac{}{}{0ex}{}{s}{y}\right)\left(\genfrac{}{}{0ex}{}{m+n-s}{n-y}\right)/\left(\genfrac{}{}{0ex}{}{m+n}{n}\right)$. Under the uniform-count predictive property, $s$ is uniform on $0,\dots ,m+n$. Dividing the joint pooled-and-batch probability by the first batch's uniform marginal gives the posterior predictive distribution, with factor $(m+1)/(m+n+1)$. Substituting $s=x+y$ in the hypergeometric expression does not give a single normalized hypergeometric mass as $y$ varies, since its population success total then varies too.

Draw $P\sim \mathrm{Beta}(\alpha ,\beta )$, $X\mid P\sim \mathrm{Binomial}(n,P)$ and $Q\mid X\sim \mathrm{Beta}(\alpha +X,\beta +n-X)$, with $\alpha ,\beta >0$ and integer $n\ge 0$. The joint density of $(P,X,Q)$ is symmetric in $P,Q$, making them exchangeable random variables with identical beta distribution marginal distributions. The posterior beta normalization constant depends on $X$ and must be retained. The correlation coefficient is $n/(n+\alpha +\beta )$: this constructs tunable dependence without altering either marginal.