Gibbs sampler

For a joint density $f_{XY}$, a two-coordinate Gibbs update draws

For Gaussian precision $\left(\begin{array}{cc}A& C\\ C& D\end{array}\right)$, a systematic update of the first coordinate followed by the second makes the centered second coordinate an AR(1) chain with coefficient $C^2/(AD)<1$. Independent conditional Gaussian noise supplies its stable innovation. This proves convergence of the Gaussian Gibbs sampler and explains slower mixing when posterior correlation has magnitude near one.

For a strictly positive posterior on finitely many binary spins, a Random-scan Gibbs sampler uses the two local posterior weights to redraw one spin. Positive full conditional distribution probabilities allow every finite sequence of spin changes, giving an irreducible Markov chain with positive self-transition probabilities.

When an observation mean depends on the neighboring spins, changing a spin affects observations centered at its neighbors. Its own observation need not involve that spin. Selecting precisely the affected factors gives a correct local full conditional distribution without recomputing the entire posterior.

Every full-conditional coordinate update satisfies detailed balance with the target distribution. Their systematic ordered composition preserves the target but need not satisfy detailed balance, since compositions of reversible kernels need not be reversible. A fixed-probability random-scan mixture remains reversible.

A blocked Gibbs sampler draws groups of coordinates jointly from their full conditional distributions. Each update preserves the joint probability distribution; dependence between blocks can still cause a long mixing time.

On a bipartite image graph, the pixels of one color are conditionally independent given the other color. Draw all black pixels from their independent full conditionals, then draw all whites conditional on the new blacks. Each color update preserves the target law, so their composition does too.

A random-scan Gibbs sampler chooses one coordinate, commonly uniformly at random, and redraws it from its complete conditional distribution while leaving all other coordinates fixed. Each coordinate update preserves the target distribution.

A single-coordinate Gibbs sampler update preserves the remaining coordinates and independently redraws the selected coordinate from its full conditional distribution. The joint measure of old and new states factors into a marginal and two identical conditional factors, making it symmetric. A fixed-probability mixture of such kernels therefore satisfies detailed balance.

A tempered Gibbs sampler changes the coordinate-selection probabilities and compensates by drawing from modified complete conditionals. Its invariant law can differ from the desired target, so expectations under the target are recovered with explicit importance weights.

The joint target density is stationary for a full Gibbs sweep, since

For independent $Z_i\mid \mu ,\omega \sim N(\mu ,{\omega }^{-1})$, a flat prior on $\mu$, and an exponential prior of rate $\lambda$ on $\omega$, the full conditionals areandwhere the gamma distribution is parametrized by shape and rate.