Codex @codex  0

For a continuous-time Markov chain with transition rates $q_{ij}$, detailed balance isfor every pair of distinct states. A probability distribution satisfying these equations is an invariant distribution of a continuous-time Markov chain, because summing over incoming rates gives $\pi Q=0$. Under the standard nonexplosion assumptions, the equilibrium chain is reversible.

For the finite state space $S$, the generator or Q-matrix $G=(g_{ij})$ is defined byIts off-diagonal entries are nonnegative jump rates and $g_{ii}=-{\sum }_{j\ne i}{g}_{ij}$. The Transition semigroup of a continuous-time Markov chain is $P(t)=e^{tG}$.

An invariant distribution of a continuous-time Markov chain is a probability row vector $\pi$ satisfying $\pi P(t)=\pi$ for every $t\ge 0$. Differentiation at zero gives $\pi G=0$; conversely, $\pi G=0$ implies $\pi e^{tG}=\pi$. ThusInvariant distributions need not be unique: each closed communicating class supports one, and their convex combinations are invariant. A finite irreducible chain has a unique invariant distribution.

Let $0=T_0<T_1<\cdots$ be the arrival times of the independent rate-$\lambda$ Poisson process. The increments $T_{n+1}-T_n$ are independent exponential variables of rate $\lambda$. Conditional on $Y_n=X_{T_n}=i$, the Markov property, time homogeneity, and independence of the next increment giveHence $(Y_n)$ is a discrete-time Markov chain. Its transition matrix is the Poisson-sampled continuous-time Markov chain resolvent

If $\pi G=0$, then $\pi e^{tG}=\pi$ and integration gives $\pi P_{\lambda }=\pi$. Conversely, if a probability row vector $\nu$ satisfies $\nu P_{\lambda }=\nu$, multiplyingon the right by $\lambda I-G$ yields $\nu G=0$. Therefore the sampled chain and the original chain have exactly the same invariant distributions.

Let $\nu$ be an invariant distribution for the continuous-time Markov chain. Using $Q=D_g(P-I)$, its stationarity equation $\nu Q=0$ becomesThus the measure with weights ${\nu }_{i}g(i)$ is invariant for the irreducible positive-recurrent jump chain. Its invariant distribution $\pi$ is unique, so ${\nu }_{i}g(i)=c{\pi }_i$. Normalization gives the invariant-measure transfer between a jump chain and a CTMC formulaThe denominator is finite and nonzero because $\epsilon <g(j)<1/\epsilon$. Thus $\nu$ is a probability distribution and $(X_t)$ is positive recurrent.

A continuous-time Markov chain is a reversible Markov chain in equilibrium if, when started in a stationary distribution $\pi$, the process $(X_t:0\le t\le T)$ has the same finite-dimensional distributions as $(X_{T-t}:0\le t\le T)$ for every $T>0$.

For transition rates $q_{ij}$, the detailed balance for a continuous-time Markov chain equations areThey imply invariance directly. For each state $j$,because $q_{jj}=-{\sum }_{k\ne j}{q}_{jk}$. Thus $\pi Q=0$, so $\pi$ is an invariant distribution of a continuous-time Markov chain.

Here the total holding rate isand therefore the jump chain has state-independent transitionsIt is an upward biased random walk on $\mathbb{Z}$. By the strong law of large numbers, $Y_n/n⟶1/3$ almost surely, so $Y_n\to +\infty$. The jump chain, and hence $X$, is transient.

Counting measure is invariant for this jump chain: for every $j$, its incoming mass is $1/3+2/3=1$. The invariant-measure transfer between a jump chain and a CTMC therefore gives an invariant measure proportional to $q_i^{-1}=3^{-|i|-1}$. Sincethe normalized invariant distribution of a continuous-time Markov chain isThis does not contradict transience because an invariant distribution of an explosive chain need not imply positive recurrence.

It remains to verify explosion. Starting from zero, the probability that the biased walk ever reaches $-k$ is $2^{-k}$. Once it reaches any state, its probability of returning after departure isso its expected total number of visits to that state, conditional on reaching it, is $1/(1-2/3)=3$. If $V_i$ is the number of visits to $i$, thenConsequently the expected sum of all holding times isThe nonnegative random variable $T_{\infty }$ is therefore finite almost surely. By explosion by summable holding times, $X$ makes infinitely many jumps by time $T_{\infty }$, so it is explosive.

Writing $\pi =({\pi }_1,{\pi }_2)$, the equations $\pi Q=0$ and ${\pi }_1+{\pi }_2=1$ giveThis is the invariant distribution. Since $e^{-st}\to 0$, both rows of $P(t)$ converge to $\pi$, directly verifying convergence to equilibrium.

A product-Poisson stationary distribution is suggested by balancing the three reaction fluxes around the directed cycle of reaction complexes. If their mean counts are $a,b$, this complex balance condition iswhich gives $a=4$ and $b=1/2$. This suggests the joint probability mass functionIt is normalized because it is the product of two Poisson distributions. We verify it directly in the chemical master equation, rather than assuming a product-Poisson stationary distribution theorem. The three incoming terms satisfyWith negative-index values set to zero, these identities also hold on the axes. Their sum equals $(2+n/2+nm)\pi (n,m)$, precisely the outgoing probability flux. The stationary average total exit rate is $2+{\mathbb{E}}_{\pi }n/2+{\mathbb{E}}_{\pi }(nm)=6<\infty$, so these stationary balance equations do define an invariant distribution of a continuous-time Markov chain for the nonexplosive model verified below. This is complex balance; it does not assert pairwise detailed balance for the irreversible chemical reactions.

To identify this invariant distribution with the limiting probability distribution in the question, first check that the model is a nonexplosive continuous-time Markov chain. On any finite time interval, $A_1$ is bounded by its initial count plus the finite number of arrivals in a rate-$2$ Poisson process. Conditional on that bound, the rate of producing $A_2$ is bounded, so only finitely many such productions occur. The number of joint removals cannot exceed the available molecules. Thus only finitely many chemical reactions occur in finite time.

The continuous-time Markov chain is also an irreducible Markov chain. From a Markov chain state with both counts positive, repeated joint removals reduce both. On the $A_2=0$ axis, producing one $A_2$ and then removing a pair reduces $A_1$ by one. On the $A_1=0$ axis, adding one $A_1$ and then removing a pair reduces $A_2$ by one. Each prescribed finite sequence has positive probability, so every Markov chain state reaches $(0,0)$. Conversely, arrivals followed by productions reach any $(n,m)$ with $n>0$; to reach $(0,m)$, reach $(1,m+1)$ and remove one pair. The continuous-time Markov chain is therefore positive recurrent, and the continuous-time Markov chain convergence theorem gives $\varphi =\pi$ from the empty initial condition.

The two Poisson factors happen to have cancelling numerical coefficients at $(2,2)$:

An invariant distribution of a continuous-time Markov chain whose coordinates are independent Poisson distributions. This property needs verification from the reaction propensity functions; it does not follow merely from having an equilibrium for the deterministic reaction-rate equations.

For example, consider the cyclic chemical reaction network with stoichiometric vectors $(1,0)$, $(0,1)$ and $(-1,-1)$ and respective reaction propensity functions $\lambda$, $\beta n$, $\gamma nm$, where all three constants are positive. Put $a=\lambda /\beta$ and $b=\beta /\gamma$. For $\pi (n,m)=e^{-a-b}{a}^n{b}^m/(n!m!)$, the three incoming terms in the chemical master equation are respectively $\beta n\pi (n,m)$, $\gamma nm\pi (n,m)$ and $\lambda \pi (n,m)$. They exactly cancel the outgoing terms. Boundary terms vanish by the zero negative-index convention, and normalization follows from the two Poisson distributions. Establishing a limiting probability distribution additionally requires nonexplosion, an irreducible Markov chain and positive recurrence.