Bayesian statistics

Bayesian statistics updates a prior distribution to a posterior distribution using likelihood.

BUGS specifies a probabilistic model through stochastic nodes, deterministic nodes and observed data, then uses Markov chain Monte Carlo to simulate its Bayesian posterior. Its normal sampling notation uses a precision parameter rather than a variance, and its gamma notation uses shape and rate.

A zeros trick implements a positive likelihood function $L(\theta )$ by adding an observed zero with Poisson distribution mean $K-\log L(\theta )$. If this mean is nonnegative throughout the parameter support, its likelihood is $e^{-K}L(\theta )$ and yields the intended Bayesian posterior. The constant $K$ must be independent of the parameter; arbitrary clipping changes the target.

Variational inference approximates a posterior distribution by minimizing a Kullback-Leibler divergence over a chosen family of probability distributions. For an unnormalized posterior density $h$ and a trial probability density function $q$, minimizing ${\mathbb{E}}_q[\log q-\log h]$ is equivalent to minimizing $\mathrm{KL}(q\Vert h/\int h)$.

A reparameterization gradient differentiates an expected value by representing the simulated random variable as a differentiable transformation of parameter-independent noise. For $X_{\eta }=g_{\eta }(\epsilon )$, the identity ${\nabla }_{\eta }\mathbb{E}[F_{\eta }(X_{\eta })]=\mathbb{E}[{\nabla }_{\eta }{F}_{\eta }(g_{\eta }(\epsilon ))]$ requires justified differentiation under the integral sign. For a normal distribution, use $X=m+e^{\rho }\epsilon$ with $\epsilon$ drawn from the standard normal distribution.

Markov chain variational inference optimizes an initial parametric probability distribution ${\nu }_{\eta }$ by minimizing $\mathrm{KL}(K^t{\nu }_{\eta }\Vert \pi )$ after a fixed number of Markov kernel updates. The best initial probability distribution can depend on $t$; Kullback-Leibler divergence contraction does not imply that optimization commutes with applying the Markov kernel.

Mean-field variational inference restricts the trial probability density function to $q(x)={\prod }_i{q}_i(x_i)$. With the other factors fixed, $q_j^{\ast }(x_j)\propto \exp ({\mathbb{E}}_{q_{-j}}\log \pi (x_j,X_{-j}))$, provided this expression has a finite positive normalization constant. Subtracting the objective at $q_j^{\ast }$ leaves $\mathrm{KL}(q_j\Vert q_j^{\ast })\ge 0$, proving the update is optimal when the terms are well defined.

Data augmentation enlarges a posterior distribution with latent variables whose removal by marginalization recovers the original posterior distribution. Convenient full conditional distributions then permit a Gibbs sampler on the enlarged space.

A hierarchical Bayesian model assigns distributions to local latent variables conditional on shared hyperparameters, then distributions to observations conditional on those latent variables. Its probabilistic graphical model records this conditional factorization, often using a plate for repeated observations. Integrating out latent variables gives the marginal likelihood of the hyperparameters. Priors described as flat on logarithms require a Jacobian determinant when densities are written in the original positive variables.

Partial pooling estimates related local parameters jointly through shared hyperparameters. Each local Bayesian posterior balances its own likelihood function against a group prior distribution, with stronger shrinkage when its data are weak or the group variation is small. It lies between forcing all local parameters to be equal and estimating them independently. Random-effects meta-analysis is an important application.

A Gamma–Poisson hierarchical model uses $Y_i\mid {\theta }_i\sim \mathrm{Poisson}(t_i{\theta }_i)$, ${\theta }_i\mid b\sim \mathrm{Gamma}(a,b)$, and $b\sim \mathrm{Gamma}(c,r)$, with gamma distributions in shape-rate convention. Conditional independence gives full conditional distributions ${\theta }_i\mid b,y\sim \mathrm{Gamma}(a+y_i,b+t_i)$ and $b\mid \theta ,y\sim \mathrm{Gamma}(c+na,r+{\sum }_i{\theta }_i)$.

For independent Poisson distributions with means $u{e}^{{\eta }_k}$ and a shape–rate Gamma distribution prior on $u$, integrating $u$ adds $a$ to the total-count exponent and $b$ to the sum of relative rates. This distinguishes gamma mixing from conditioning on the total count, which gives a multinomial distribution.

An observed astronomical colour is the sum of a normal intrinsic colour, a nonnegative dust contribution with an exponential distribution, and normal measurement noise. With latent $C_s\sim N(\mu ,v)$, $E_s\sim \mathrm{Exp}(\text{mean }\tau )$ and $O_s\mid C_s,E_s\sim N(C_s+E_s,r_s)$, the Gibbs sampler conditionals for $C_s$ are normal and those for $E_s$ are truncated normal distributions. A flat prior on $\mu$ and inverse-gamma priors on $v,\tau$ give inverse-gamma conditional updates for the two scales. Log-flat priors on both scales instead make the joint posterior improper when all $r_s>0$, despite formally proper full conditionals at generic latent states: the marginal observed likelihood has a positive limit at either zero-scale boundary. Proper positive-scale priors restore a genuine joint posterior.

A maximum a posteriori estimate maximizes the posterior density of a parameter or latent configuration. Equivalently, it maximizes the sum of the log likelihood and log prior, relative to the chosen reference measure.

A credible interval is an interval whose posterior probability equals a prescribed level. Conditional on the observed data and the Bayesian model, the unknown parameter lies in the interval with that probability.

An empirical Bayes method estimates prior hyperparameters from the marginal distribution of the observed data and then uses the resulting fitted prior in Bayesian inference.

A maximum marginal likelihood estimator chooses a hyperparameter by maximizing the data density obtained after integrating the model parameter against its prior distribution.

A hyperparameter controls a family of prior distributions or statistical models and is fixed, estimated, or assigned a further prior at a higher level of a hierarchical model.

A Dirichlet process is a distribution over probability measures such that the masses assigned to every finite measurable partition have a Dirichlet distribution. Its draws are almost surely discrete.

A Dirichlet prior with parameters $({\alpha }_1,\dots ,{\alpha }_K)$ combined with categorical labels having counts $(N_1,\dots ,N_K)$ gives a Dirichlet posterior with parameters $({\alpha }_1+N_1,\dots ,{\alpha }_K+N_K)$.

A prior probability describes uncertainty about an event or parameter before the current observation is incorporated.

A prior density is the probability density function of a continuous prior distribution with respect to a specified reference measure. A proper prior density is nonnegative and integrates to one. Prior-density factors enter Bayesian posterior and RJ-MCMC acceptance formulas along with the likelihood and model prior probabilities.

A continuous spike-and-slab prior mixes a narrow continuous probability distribution around zero with a much wider component. It permits practical effect selection while retaining a continuous posterior density. Its narrow component is not the exact-zero mass of a point-null mixture prior. A latent component indicator aids computation; a wide-component draw can still have small magnitude, so posterior effect thresholds and component membership answer different questions.

A uniform prior on a finite-volume parameter region assigns constant probability density there and zero outside. On $[0,1]$ it is $\mathrm{Beta}(1,1)$, giving a Beta distribution posterior after binomial distribution data. Uniformity depends on the coordinate: a nonlinear reparametrization introduces a Jacobian and generally does not preserve a uniform density.

The posterior density is proportional to likelihood times prior density.

A posterior variance is the variance of a parameter under its Bayesian posterior, namely $\mathbb{E}[(\theta -\mathbb{E}[\theta \mid y]{)}^2\mid y]$ when finite. It describes remaining uncertainty after observing data and is distinct from the repeated-sampling variance of a fitted estimator.

The distribution of a new or missing quantity integrates its model over the Bayesian posterior: $p(y^{\ast }\mid y)=\int p(y^{\ast }\mid \varphi ,y)p(\varphi \mid y)d\varphi$. Both parameter and residual uncertainty should enter predictive draws used for multiple imputation.

A posterior predictive check compares a summary of the observed data with the same summary in replicated data drawn from a model's posterior predictive distribution. The replication must match the target: replicating an existing group's observations conditional on its effect differs from predicting a new group with a newly drawn effect. Such checks expose mismatches in dispersion, tails or other features; using the observations to fit and check the model means they are not automatically calibrated frequentist tests.

For independent observations $y_s\sim N(\beta +d_s,V_s)$, put $a_s=V_s^{-1}$, $S={\sum }_s{a}_s$, $r_s=y_s-d_s$, $\bar{r}=S^{-1}{\sum }_s{a}_s{r}_s$ and $Q={\sum }_s{a}_s(r_s-\bar{r}{)}^2$. Integrating the likelihood function against a flat improper prior on $\beta$ gives, up to the arbitrary prior constant,Indeed ${\sum }_s{a}_s(r_s-\beta {)}^2=Q+S(\beta -\bar{r}{)}^2$, and the remaining one-dimensional Gaussian integral is $\sqrt{2\pi /S}$. The conditional Bayesian posterior of $\beta$ is $N(\bar{r},S^{-1})$.

A posterior predictive probability averages a future-event probability over the Bayesian posterior. For a Bernoulli distribution with unknown parameter $p$, it is $\mathbb{E}[p\mid \mathcal{D}]$. Under Beta-binomial conjugacy with posterior $\mathrm{Beta}(a,b)$, the predictive success probability is $a/(a+b)$.

A posterior density is the density of a posterior distribution with respect to a chosen reference measure.

The log-posterior is the logarithm of a positive posterior density. Its gradient is the posterior score control variate when boundary terms vanish. An additive constant independent of the parameter does not affect optimization or differentiation.

The posterior mean is the expected value of a parameter under its posterior distribution; under squared-error loss it is the Bayes estimator.

A posterior probability is the probability assigned to an event by the posterior distribution after conditioning on the observed data.

A point-null mixture prior assigns positive mass to one parameter value and distributes the remaining mass continuously over alternatives. Posterior point mass is obtained by dividing the null component of the marginal density by the full marginal density.

For $\overline{X}\mid \mu \sim N(\mu ,1/n)$ and an equal mixture of $\mu =0$ and $\mu \sim N(0,{\tau }^2)$,

For the equal point-null mixture,

With a fixed diffuse alternative prior, a frequentist p-value and the posterior probability of a point null can order evidence differently. For $n=100$ and $\tau =1$, the p-value exceeds the null posterior near zero, while sufficiently far in the tails the null posterior exceeds the p-value because it has a slightly slower Gaussian decay.

An improper prior is a nonnegative prior kernel with infinite total mass. It can still produce a proper posterior after multiplication by the likelihood and normalization.

The measure $d\sigma /\sigma$ is invariant under multiplication by every positive constant. In the logarithmic coordinate it is Lebesgue measure. It cannot be normalized over $(0,\infty )$, although a bounded truncation is a log-uniform distribution.

A random-effect model with positive marginal likelihood at scale zero cannot yield a proper posterior distribution under a prior proportional to the reciprocal scale near zero. On a compact set of the other parameters, continuity and positivity give a positive likelihood lower bound, while ${\int }_0^{\epsilon }d\lambda /\lambda =\infty$. Individual full conditional distributions can still be proper, so their existence alone does not justify a Gibbs sampler with a probability distribution as its joint target.

A posterior kernel $L(\theta )\pi (\theta )$ defines a posterior distribution exactly when its integral is finite and nonzero. Proper full conditional densities do not suffice. For example, if a known positive measurement variance makes a marginal normal likelihood approach a strictly positive limit as a latent variance $v$ tends to zero, the log-flat prior $dv/v$ gives infinite posterior mass at that boundary. The same problem occurs for a latent exponential scale $\tau$ with $d\tau /\tau$ when the observed likelihood has a positive limit as the exponential contribution vanishes.

For a Poisson observation of mean $\theta$ and a gamma prior with shape $\alpha$ and rate $\lambda$, the posterior is gamma with shape $\alpha +X$ and rate $\lambda +1$.

Given $\Theta$, independent observations $Y_j$ have Poisson distribution with means $m_j\Theta$ for known exposures. A gamma distribution prior of shape $\alpha$ and rate $\beta$ yields a gamma posterior with shape $\alpha +{\sum }_j{Y}_j$ and rate $\beta +{\sum }_j{m}_j$. The likelihood kernel is ${\theta }^{{\sum }_j{Y}_j}{e}^{-\theta {\sum }_j{m}_j}$. Thus the posterior mean depends on total claims and total exposure, not the number of periods alone.

A gamma prior with shape $\alpha$ and rate $\beta$ combined with $n$ independent observations from an exponential distribution of rate $\theta$ gives the posteriorUnder squared-error loss, the Bayes estimator is its posterior mean $(\alpha +n)/(\beta +{\sum }_i{X}_i)$.

A beta prior $\mathrm{Beta}(\alpha ,\beta )$ combined with $s$ Bernoulli successes and $f$ failures gives posterior $\mathrm{Beta}(\alpha +s,\beta +f)$.

Draw $P\sim \mathrm{Beta}(\alpha ,\beta )$, $X\mid P\sim \mathrm{Binomial}(n,P)$ and $Q\mid X\sim \mathrm{Beta}(\alpha +X,\beta +n-X)$, with $\alpha ,\beta >0$ and integer $n\ge 0$. The joint density of $(P,X,Q)$ is symmetric in $P,Q$, making them exchangeable random variables with identical Beta distribution marginal distributions. The posterior beta normalization constant depends on $X$ and must be retained. The correlation coefficient is $n/(n+\alpha +\beta )$: this constructs tunable dependence without altering either marginal.

A Bayes estimator minimizes posterior expected loss.

A Bayes decision rule maximizes posterior expected utility, equivalently minimizes posterior expected loss, separately at each observed data value.

Under squared error loss, the Bayes estimator is the posterior mean.

For $X\mid \theta \sim N_p(\theta ,I_p)$ and $\theta \sim N_p(0,c^2{I}_p)$, the posterior isThe Bayes estimator under quadratic loss is the posterior mean and its Bayes risk is the trace of the posterior covariance, $p{c}^2/(1+c^2)$.

For loss $L(\theta ,a)=q(\theta )(\theta -a{)}^2$ with $q(\theta )>0$, the Bayes estimator is the mean under the posterior reweighted by $q$:

Under $L(d,\theta )={\theta }^{-1}(\theta -d{)}^2$, the posterior expected loss is minimized atprovided the conditional inverse moment is finite and positive.

Posterior expected loss averages the loss over the posterior and is minimized pointwise in the data.

Markov chain Monte Carlo constructs an ergodic Markov chain whose stationary distribution is the target law and uses its post-burn-in states as approximate dependent samples.

Reversible-jump Markov chain Monte Carlo samples a posterior on a disjoint union of model-specific parameter spaces. A dimension-matching bijection between old parameters plus auxiliary variables and new parameters plus reverse auxiliary variables supplies a reversible proposal. Its acceptance ratio includes posterior-density, model-selection, proposal-density, and Jacobian determinant factors. The equal-dimension reversible-jump acceptance probability is the ordinary density-based special case.

A pseudo-prior is a proper density assigned to a parameter that is inactive under a particular model. Integrating it out leaves that model's likelihood and marginal evidence unchanged. Its choice affects movement between model states, but not the intended marginal posterior over models.

A pseudo-prior can make model parameter vectors have equal dimensions, allowing identity-matched RJ-MCMC moves. If active and inactive coordinates share the same prior density in the two augmented targets, these factors cancel in model-switch ratios. Within-model updates must preserve the augmented target, including the inactive coordinate's pseudo-prior law.

Compare independent Poisson means with one shared mean under proper shape-rate gamma distribution priors. Adding the second mean as an inactive parameter under the shared-mean model gives a simple identity model switch. With equal model priors and identical active/inactive priors, its ratio from separate to shared means is $e^{{\gamma }_2-{\gamma }_1}({\gamma }_1/{\gamma }_2{)}^{x_2}$. Poisson-gamma conjugacy supplies exact within-model updates, and integrating the gamma kernels supplies an independent Bayes factor benchmark.

For unnormalized model posterior densities $h_j$, model-selection probabilities $s_{jk}$ and parameter-proposal densities $q_{jk}$, accept a proposal $(j,\theta )\to (k,{\theta }^{\prime })$ with the minimum of one and $h_k({\theta }^{\prime })s_{kj}({\theta }^{\prime })q_{kj}(\theta \mid {\theta }^{\prime })/[h_j(\theta )s_{jk}(\theta )q_{jk}({\theta }^{\prime }\mid \theta )]$. A deterministic matching-map formulation instead includes its explicit Jacobian determinant; it must not be counted again when already absorbed into a direct proposal density.

The hit-and-run sampler moves within a full-dimensional compact convex set by choosing a uniform direction, then a uniform point on the chord through the current point in that direction. Its invariant law is the uniform distribution on the set. Starting in the interior keeps the chain in the interior almost surely. To define absolutely continuous transitions also at boundary points, redraw directions whose chords have zero length. Without this convention, a corner can have a positive holding atom.

For a planar hit-and-run sampler, let $\ell (x,u)$ be the chord length and $a(x)$ the fraction of directions with positive chord length. With zero-length chords redrawn, the probability density function isfor almost every $y$ in the body. Interior points have $a(x)=1$. The two signed representations of a direction contribute the factor two in the polar-coordinate calculation. If the body has diameter $D$, a version of this density is bounded below by $1/(\pi D^2)$. This is a global Doeblin condition after normalization by the body's area.

Several chains from dispersed initial states, trace plots, autocorrelation, effective sample sizes, and the rank-normalized split potential scale reduction factor $\hat{R}$ assess mixing and disagreement between chains. An $\hat{R}$ close to one is useful evidence but cannot prove convergence or prove posterior propriety. Comparing chains that explore different modes is especially important; monitor each scientifically important observable and its Monte Carlo uncertainty. A long apparent plateau can also occur when an improper-target sampler is drifting toward a boundary.

Thinning keeps every $L$th state of a Markov chain. The retained chain has transition kernel $K^L$ and the same stationary distribution. It is generally still dependent, and reducing correlation between retained draws does not by itself improve precision per original transition. Keep all post-warmup draws unless storage or later processing requires thinning; assess precision using the effective sample size of a Markov chain.

A Markov kernel is geometrically ergodic when its iterates converge to stationarity in total variation at a geometric rate, with a finite state-dependent prefactor.

For an aperiodic positive Harris recurrent Markov chain with geometric ergodicity, an observable with stationary moment $\pi (|h{|}^{2+\delta })<\infty$ for some $\delta >0$ satisfies a central limit theorem. Its asymptotic variance is ${\gamma }_h(0)+2{\sum }_{k\ge 1}{\gamma }_h(k)$, with stationary covariances. Ergodicity alone does not guarantee this theorem.

Uniform geometric ergodicity strengthens geometric ergodicity by bounding the prefactor uniformly over the starting state. If $P(x,\cdot )\ge \epsilon \mu (\cdot )$ globally and $\mu$ is invariant, thenWrite $P=\epsilon \Pi +(1-\epsilon )R$, where $\Pi$ draws independently from $\mu$, to obtain the bound.

Applying a Markov kernel to two initial laws cannot increase their total variation distance, even for the distribution of an entire subsequent finite trajectory. If a bounded summand has absolute value at most $B$, the variance of a length-$N$ path sum differs between these laws by at most $6B^2{N}^2$ times their total variation distance. Thus a geometrically long enough burn-in makes this difference negligible relative to $N$.

For a reversible Markov chain with invariant law $\mu$, its Markov kernel acts as a self-adjoint operator $P$ on $L^2(\mu )$. An absolute spectral gap means that $\Vert P\Vert$ on the mean-zero functions is at most $r<1$. Geometric ergodicity implies this gap for a reversible chain. Indeed, for bounded mean-zero $g$ supported where the geometric prefactor is bounded, the total-variation estimate gives $⟨g,P^{2n}g⟩\le C_g{r}^{2n}$. The spectral theorem for normal operators puts its spectral measure in $[-r,r]$. Such test functions are dense in the mean-zero subspace.

If $\Vert P{\Vert }_{L_0^2(\mu )}\le r<1$, a stationary reversible Markov chain satisfiesConsequently the variance of a sum of $N$ consecutive values is at most $N(1+r)/(1-r)$ times the stationary variance of one value. This bounds the Markov chain Monte Carlo asymptotic variance uniformly over square-integrable functions.

For a stationary ergodic Markov chain with invariant distribution $\pi$ and an integrable function $h$, the sample mean $n^{-1}{\sum }_{t=1}^{n}h(X_t)$ converges almost surely to $\int hd\pi$. Geometric ergodicity supplies standard sufficient conditions for this conclusion and stronger limit theorems.

A minorization condition gives a common component of transition laws: $K^m(x,\cdot )\ge \alpha \nu (\cdot )$ for every $x$ in a specified set, some $m\ge 1$, $\alpha >0$, and a probability measure $\nu$.

A Doeblin condition is a minorization of the whole state space. It implies uniform geometric convergence in total variation distance.

A set $A$ is $\alpha$-small when some iterate satisfies $K^m(x,\cdot )\ge \alpha \nu (\cdot )$ for every $x\in A$ and some probability measure $\nu$.

A geometric drift $KV\le \lambda V+b{1}_A$ with $\lambda <1$, together with a small-set minorisation on $A$ and irreducibility and aperiodicity, implies geometric ergodicity.

A geometric drift condition requires a measurable $V\ge 1$, a small set $C$, constants $\lambda <1$ and $b<\infty$, andTogether with irreducibility and aperiodicity, it implies geometric ergodicity. A global Doeblin condition allows $C$ to be the entire state space and $V=1$.

Suppose a Markov kernel satisfies $PV(x)\le M<\infty$ for a measurable $V\ge 1$. For $0<\rho <1$ and $R>M/\rho$, the sublevel set $C=\{V\le R\}$ gives $PV\le \rho V+M{1}_C$. If $C$ is a small set, this is a geometric drift condition. The hypotheses of an irreducible Markov chain and an aperiodic Markov chain are also required for the usual geometric ergodicity theorem.

For a stationary chain, the asymptotic variance of the sample mean is ${\mathrm{Var}}_{\pi }\psi +2{\sum }_{k\ge 1}{\mathrm{Cov}}_{\pi }(\psi (X_0),\psi (X_k))$ when the series converges.

For a reversible kernel $K$, the inequality ${\mathrm{Var}}_{\pi }(f)\le C{\mathcal{E}}_{K^2}(f)$ is equivalent to geometric contraction of variance under iteration of $K$.

One Markov kernel dominates another in Peskun ordering when it assigns at least as much transition probability to every measurable set not containing the current state. For reversible kernels this increases Dirichlet energy and cannot decrease the spectral gap.

Hamiltonian Monte Carlo introduces an auxiliary Gaussian momentum, approximately follows Hamiltonian dynamics by a reversible volume-preserving integrator, and applies a Metropolis correction.

Surrogate Hamiltonian Monte Carlo uses an auxiliary smooth probability density function $\nu$ to generate trajectories but applies a Metropolis–Hastings acceptance probability using the intended target $\mu$. For Gaussian momenta and an involutive Metropolis proposal generated by flipped leapfrog integration, the acceptance probability is $\min (1,\mu (x^{\prime })e^{-\Vert p^{\prime }{\Vert }^2/2}/[\mu (x)e^{-\Vert p{\Vert }^2/2}])$. Only the surrogate gradient is required during the trajectory; target density evaluations are still required at the endpoints.

The Metropolis–Hastings algorithm proposes $y$ from $q(x,\cdot )$ at state $x$ and accepts it with probabilityThis acceptance rule enforces detailed balance with the target density $\pi$.

A pseudo-marginal Metropolis–Hastings algorithm replaces an intractable likelihood function by a nonnegative unbiased likelihood estimator and includes its auxiliary randomness in the state. If $\mathbb{E}[\hat{L}(\theta ,U)]=L(\theta )$ for $U\sim m_{\theta }$, the extended target is proportional to $p(\theta )\hat{L}(\theta ,u)m_{\theta }(u)$. An independence proposal for new auxiliary randomness cancels the $m_{\theta }$ factors in the Metropolis–Hastings acceptance probability. On rejection the old estimate must be retained. Integrating the extended target gives the desired marginal posterior distribution exactly.

An unbiased likelihood estimator has expectation equal to the likelihood function at each parameter. A pseudo-marginal algorithm requires this estimator to be nonnegative. Importance sampling provides $\hat{L}=M^{-1}{\sum }_j\ell (F_j)p_{\theta }(F_j)/g_{\theta }(F_j)$ for independent random variables $F_j\sim g_{\theta }$, provided the proposal covers the target support. Relative variance affects the mixing time even though unbiasedness guarantees the correct marginal target.

For target density $\pi$ and proposal density $q$, the Metropolis–Hastings acceptance probability is $1\wedge \{\pi (y)q(y,x)/(\pi (x)q(x,y))\}$.

At the current state $x$, a Metropolis–Hastings algorithm draws a candidate state $y$ from its proposal distribution $q(\cdot \mid x)$. The Metropolis–Hastings acceptance probability corrects the proposal's bias so that the target distribution remains invariant.

An involutive Metropolis proposal uses a deterministic bijection $S$ satisfying $S^2=I$. For a volume-preserving $S$ and target probability density function $\rho$, accept $S(z)$ with probability $\min (1,\rho (S(z))/\rho (z))$. The identity $\rho (z)\alpha (z)=\min (\rho (z),\rho (S(z)))$ and a change of variables under $S$ prove detailed balance. A non-unit Jacobian determinant must be included when volume is not preserved.

For a proposal $Y=AX+Z$, with isotropic $Z\sim N(0,I)$ and an independent random orthogonal matrix $A$, invariance of the distribution of $A$ under transpose makes the proposal distribution symmetric. Indeed $|y-Ax|=|x-A^{T}y|$, and averaging the isotropic Gaussian density over an inversion-invariant law yields $q(x,y)=q(y,x)$. The Metropolis–Hastings algorithm therefore accepts with the target density ratio alone.

Let $0<\beta \le 1$, let $X\in {\mathbb{R}}^p$, and proposeThe proposal distribution is $N(\sqrt{1-{\beta }^2}X,{\beta }^2{I}_p)$. If $X\sim N(0,I_p)$ independently of $Z$, then $(X,Y)$ is a jointly multivariate normal distribution invariant under exchanging $X$ and $Y$, because both random vectors have covariance matrix $I_p$ and their cross-covariance matrices are both $\sqrt{1-{\beta }^2}{I}_p$. Consequently its density $q$ satisfieswhere $\varphi$ is the standard-normal density. Thus the proposal is reversible with respect to the standard normal distribution.

The preconditioned Crank–Nicolson algorithm, or pCN algorithm, uses a Gaussian autoregressive proposal reversible with respect to a standard normal distribution in a Metropolis–Hastings algorithm. When the target density is a likelihood times the proposal's invariant Gaussian prior density, the Gaussian factors cancel from the Metropolis–Hastings acceptance probability, leaving the likelihood ratio.

Random-walk Metropolis proposes a symmetric increment from the current state and accepts a proposal $y$ from $x$ with probability $\min \{1,\pi (y)/\pi (x)\}$.

An independence Metropolis–Hastings algorithm proposes from a fixed density $q$ independent of the current state. If ${\sup }_x\pi (x)/q(x)<\infty$, it is uniformly and hence geometrically ergodic.

The effective sample size discounts a correlated chain's raw length by its integrated autocorrelation time.

For a stationary Markov chain and a scalar observable with summable autocorrelation function $\rho (k)$, the asymptotic variance of its sample mean is $\mathrm{Var}(h){\tau }_{\mathrm{int}}/n$. Thus the effective sample size of a Markov chain is approximately $n/{\tau }_{\mathrm{int}}$. Different observables can have very different autocorrelation times.

For a joint density $f_{XY}$, a two-coordinate Gibbs update draws

For Gaussian precision $\left(\begin{array}{cc}A& C\\ C& D\end{array}\right)$, a systematic update of the first coordinate followed by the second makes the centered second coordinate an AR(1) chain with coefficient $C^2/(AD)<1$. Independent conditional Gaussian noise supplies its stable innovation. This proves convergence of the Gaussian Gibbs sampler and explains slower mixing when posterior correlation has magnitude near one.

For a strictly positive posterior on finitely many binary spins, a Random-scan Gibbs sampler uses the two local posterior weights to redraw one spin. Positive full conditional distribution probabilities allow every finite sequence of spin changes, giving an irreducible Markov chain with positive self-transition probabilities.

When an observation mean depends on the neighboring spins, changing a spin affects observations centered at its neighbors. Its own observation need not involve that spin. Selecting precisely the affected factors gives a correct local full conditional distribution without recomputing the entire posterior.

Every full-conditional coordinate update satisfies detailed balance with the target distribution. Their systematic ordered composition preserves the target but need not satisfy detailed balance, since compositions of reversible kernels need not be reversible. A fixed-probability random-scan mixture remains reversible.

A blocked Gibbs sampler draws groups of coordinates jointly from their full conditional distributions. Each update preserves the joint probability distribution; dependence between blocks can still cause a long mixing time.