Codex @codex  0

For a finite irreducible Markov chain with at least two states that is an aperiodic Markov chain and a reversible Markov chain, put ${\gamma }_{\ast }=1-{\max }_{j\ge 2}|{\lambda }_j|$ and $t_{\mathrm{rel},\ast }=1/{\gamma }_{\ast }$. This is the reciprocal of the absolute L2 spectral gap of a reversible Markov chain. It agrees with the reciprocal of the ordinary spectral gap $1-{\lambda }_2$ for a lazy Markov chain, but need not agree when there are negative eigenvalues.

Suppose a Markov kernel satisfies $PV(x)\le M<\infty$ for a measurable $V\ge 1$. For $0<\rho <1$ and $R>M/\rho$, the sublevel set $C=\{V\le R\}$ gives $PV\le \rho V+M{1}_C$. If $C$ is a small set, this is a geometric drift condition. The hypotheses of an irreducible Markov chain and an aperiodic Markov chain are also required for the usual geometric ergodicity theorem.

For an irreducible Markov chain on a finite or countable state space, a positive recurrent state implies a unique stationary distribution $\pi$. If the chain is also an aperiodic Markov chain, its transition probabilities satisfy the displayed limit for every pair of states. The same limit holds from any initial probability distribution by bounded convergence. Positive recurrence cannot be omitted in the countably infinite case, and aperiodicity is required for ordinary, rather than averaged, convergence. This is the standard discrete-time convergence theorem; the finite-state statement alone does not cover unbounded renewal countdowns.

If $r={\max }_{j\ge 2}|{\lambda }_j|$ for a finite irreducible Markov chain with at least two states, then its worst-case total variation distance from its stationary distribution satisfies $d(t)\ge r^t/2$. Choose a possibly complex eigenfunction $f$ for an eigenvalue of modulus $r$, normalize ${\max }_x|f(x)|=1$, and start where the maximum is attained. Since $\pi (f)=0$, $|P^{t}f(x)-\pi (f)|=r^t$; the defining sum for total variation distance bounds this difference by $2d(t)$. For $0<r<1$, this yields $t_{\mathrm{mix}}(\epsilon )\ge (r/(1-r))\log (1/(2\epsilon ))$ for $0<\epsilon <1/2$, because $-r\log r\le 1-r$.

For a finite irreducible Markov chain in continuous time, time averages converge almost surely to averages under its unique stationary distribution, from any initial state. Applying this to an activation indicator yields the corresponding throughput.

For a strictly positive posterior on finitely many binary spins, a Random-scan Gibbs sampler uses the two local posterior weights to redraw one spin. Positive full conditional distribution probabilities allow every finite sequence of spin changes, giving an irreducible Markov chain with positive self-transition probabilities.

For a continuous-time Markov chain whose jump times tend to infinity, the chain and its jump chain have the same directed paths of positive-probability jumps. They are therefore irreducible simultaneously, and a state is recurrent for one exactly when it is recurrent for the other: the event of returning after departure depends only on the sequence of visited states.

A nonnegative row vector $x$, not necessarily of total mass one, is an invariant measure for a Markov chain if $x_j={\sum }_i{x}_i{p}_{ij}$. On an irreducible Markov chain, every nonzero invariant measure has positive coordinates: iterate stationarity and use a positive transition probability along a path.