Invariant distribution of a continuous-time Markov chain

For a finite-state chain with transition semigroup $P(t)=e^{tQ}$, a probability row vector $\pi$ is invariant when $\pi P(t)=\pi$ for every $t\ge 0$. Equivalently,

A countable nonexplosive continuous-time Markov chain which is an irreducible Markov chain and has an invariant distribution $\pi$ is positive recurrent; its transition probabilities tend to $\pi$ from every initial Markov chain state.

To see why continuous time removes periodicity, fix $h>0$. Every finite sequence of Markov chain states connected by positive transition intensities has positive probability of finishing before $h$, followed by no further transition until $h$. Consequently the transition matrix $P_h$ has positive entries between every pair of communicating Markov chain states, including its diagonal. It is an irreducible Markov chain and an aperiodic Markov chain with stationary distribution $\pi$. The countable-state Markov chain convergence theorem gives convergence at times $nh$, also in total variation distance. For $t=nh+r$, $0\le r<h$, applying $P_r$ contracts total variation distance and preserves $\pi$, giving the full continuous-time limit.

An invariant distribution of a continuous-time Markov chain whose coordinates are independent Poisson distributions. This property needs verification from the reaction propensity functions; it does not follow merely from having an equilibrium for the deterministic reaction-rate equations.

For example, consider the cyclic chemical reaction network with stoichiometric vectors $(1,0)$, $(0,1)$ and $(-1,-1)$ and respective reaction propensity functions $\lambda$, $\beta n$, $\gamma nm$, where all three constants are positive. Put $a=\lambda /\beta$ and $b=\beta /\gamma$. For $\pi (n,m)=e^{-a-b}{a}^n{b}^m/(n!m!)$, the three incoming terms in the chemical master equation are respectively $\beta n\pi (n,m)$, $\gamma nm\pi (n,m)$ and $\lambda \pi (n,m)$. They exactly cancel the outgoing terms. Boundary terms vanish by the zero negative-index convention, and normalization follows from the two Poisson distributions. Establishing a limiting probability distribution additionally requires nonexplosion, an irreducible Markov chain and positive recurrence.

Every closed communicating class of a finite Markov chain has its own invariant probability distribution. All invariant distributions are convex combinations of those class distributions. In particular, a finite irreducible chain has a unique invariant distribution, while a reducible chain may have many.

An explosive chain can have a summable solution of $\pi Q=0$ despite being transient. Nonexplosion is needed for the usual equivalence between invariant distributions and positive recurrence.