Bayesian posterior

The posterior density is proportional to likelihood times prior density.

A posterior variance is the variance of a parameter under its Bayesian posterior, namely $\mathbb{E}[(\theta -\mathbb{E}[\theta \mid y]{)}^2\mid y]$ when finite. It describes remaining uncertainty after observing data and is distinct from the repeated-sampling variance of a fitted estimator.

The distribution of a new or missing quantity integrates its model over the Bayesian posterior: $p(y^{\ast }\mid y)=\int p(y^{\ast }\mid \varphi ,y)p(\varphi \mid y)d\varphi$. Both parameter and residual uncertainty should enter predictive draws used for multiple imputation.

A posterior predictive check compares a summary of the observed data with the same summary in replicated data drawn from a model's posterior predictive distribution. The replication must match the target: replicating an existing group's observations conditional on its effect differs from predicting a new group with a newly drawn effect. Such checks expose mismatches in dispersion, tails or other features; using the observations to fit and check the model means they are not automatically calibrated frequentist tests.

For independent observations $y_s\sim N(\beta +d_s,V_s)$, put $a_s=V_s^{-1}$, $S={\sum }_s{a}_s$, $r_s=y_s-d_s$, $\bar{r}=S^{-1}{\sum }_s{a}_s{r}_s$ and $Q={\sum }_s{a}_s(r_s-\bar{r}{)}^2$. Integrating the likelihood function against a flat improper prior on $\beta$ gives, up to the arbitrary prior constant,Indeed ${\sum }_s{a}_s(r_s-\beta {)}^2=Q+S(\beta -\bar{r}{)}^2$, and the remaining one-dimensional Gaussian integral is $\sqrt{2\pi /S}$. The conditional Bayesian posterior of $\beta$ is $N(\bar{r},S^{-1})$.

A posterior predictive probability averages a future-event probability over the Bayesian posterior. For a Bernoulli distribution with unknown parameter $p$, it is $\mathbb{E}[p\mid \mathcal{D}]$. Under Beta-binomial conjugacy with posterior $\mathrm{Beta}(a,b)$, the predictive success probability is $a/(a+b)$.

A posterior density is the density of a posterior distribution with respect to a chosen reference measure.

The log-posterior is the logarithm of a positive posterior density. Its gradient is the posterior score control variate when boundary terms vanish. An additive constant independent of the parameter does not affect optimization or differentiation.

The posterior mean is the expected value of a parameter under its posterior distribution; under squared-error loss it is the Bayes estimator.

A posterior probability is the probability assigned to an event by the posterior distribution after conditioning on the observed data.

A point-null mixture prior assigns positive mass to one parameter value and distributes the remaining mass continuously over alternatives. Posterior point mass is obtained by dividing the null component of the marginal density by the full marginal density.

For $\overline{X}\mid \mu \sim N(\mu ,1/n)$ and an equal mixture of $\mu =0$ and $\mu \sim N(0,{\tau }^2)$,

For the equal point-null mixture,

With a fixed diffuse alternative prior, a frequentist p-value and the posterior probability of a point null can order evidence differently. For $n=100$ and $\tau =1$, the p-value exceeds the null posterior near zero, while sufficiently far in the tails the null posterior exceeds the p-value because it has a slightly slower Gaussian decay.

An improper prior is a nonnegative prior kernel with infinite total mass. It can still produce a proper posterior after multiplication by the likelihood and normalization.

The measure $d\sigma /\sigma$ is invariant under multiplication by every positive constant. In the logarithmic coordinate it is Lebesgue measure. It cannot be normalized over $(0,\infty )$, although a bounded truncation is a log-uniform distribution.

A random-effect model with positive marginal likelihood at scale zero cannot yield a proper posterior distribution under a prior proportional to the reciprocal scale near zero. On a compact set of the other parameters, continuity and positivity give a positive likelihood lower bound, while ${\int }_0^{\epsilon }d\lambda /\lambda =\infty$. Individual full conditional distributions can still be proper, so their existence alone does not justify a Gibbs sampler with a probability distribution as its joint target.

A posterior kernel $L(\theta )\pi (\theta )$ defines a posterior distribution exactly when its integral is finite and nonzero. Proper full conditional densities do not suffice. For example, if a known positive measurement variance makes a marginal normal likelihood approach a strictly positive limit as a latent variance $v$ tends to zero, the log-flat prior $dv/v$ gives infinite posterior mass at that boundary. The same problem occurs for a latent exponential scale $\tau$ with $d\tau /\tau$ when the observed likelihood has a positive limit as the exponential contribution vanishes.

For a Poisson observation of mean $\theta$ and a gamma prior with shape $\alpha$ and rate $\lambda$, the posterior is gamma with shape $\alpha +X$ and rate $\lambda +1$.

Given $\Theta$, independent observations $Y_j$ have Poisson distribution with means $m_j\Theta$ for known exposures. A gamma distribution prior of shape $\alpha$ and rate $\beta$ yields a gamma posterior with shape $\alpha +{\sum }_j{Y}_j$ and rate $\beta +{\sum }_j{m}_j$. The likelihood kernel is ${\theta }^{{\sum }_j{Y}_j}{e}^{-\theta {\sum }_j{m}_j}$. Thus the posterior mean depends on total claims and total exposure, not the number of periods alone.

A gamma prior with shape $\alpha$ and rate $\beta$ combined with $n$ independent observations from an exponential distribution of rate $\theta$ gives the posteriorUnder squared-error loss, the Bayes estimator is its posterior mean $(\alpha +n)/(\beta +{\sum }_i{X}_i)$.

A beta prior $\mathrm{Beta}(\alpha ,\beta )$ combined with $s$ Bernoulli successes and $f$ failures gives posterior $\mathrm{Beta}(\alpha +s,\beta +f)$.

Draw $P\sim \mathrm{Beta}(\alpha ,\beta )$, $X\mid P\sim \mathrm{Binomial}(n,P)$ and $Q\mid X\sim \mathrm{Beta}(\alpha +X,\beta +n-X)$, with $\alpha ,\beta >0$ and integer $n\ge 0$. The joint density of $(P,X,Q)$ is symmetric in $P,Q$, making them exchangeable random variables with identical Beta distribution marginal distributions. The posterior beta normalization constant depends on $X$ and must be retained. The correlation coefficient is $n/(n+\alpha +\beta )$: this constructs tunable dependence without altering either marginal.