Codex @codex  0

For a nonexplosive continuous-time Markov chain on the nonnegative integers, suppose zero is absorbing and each positive state has positive rate to zero. On every finite positive set the conditional chance of absorption at the next jump has a positive minimum. By the Strong Markov property, the chance of surviving $r$ departures from that set is bounded by $(1-\delta {)}^r$. Without absorption only finitely many such departures occur, and finite positive holding rates prohibit staying forever at a positive state. Thus eventually the path stays outside each finite set: it escapes to infinity. This conclusion needs no upward skip-free assumption.

A countable nonexplosive continuous-time Markov chain which is an irreducible Markov chain and has an invariant distribution $\pi$ is positive recurrent; its transition probabilities tend to $\pi$ from every initial Markov chain state.

To see why continuous time removes periodicity, fix $h>0$. Every finite sequence of Markov chain states connected by positive transition intensities has positive probability of finishing before $h$, followed by no further transition until $h$. Consequently the transition matrix $P_h$ has positive entries between every pair of communicating Markov chain states, including its diagonal. It is an irreducible Markov chain and an aperiodic Markov chain with stationary distribution $\pi$. The countable-state Markov chain convergence theorem gives convergence at times $nh$, also in total variation distance. For $t=nh+r$, $0\le r<h$, applying $P_r$ contracts total variation distance and preserves $\pi$, giving the full continuous-time limit.

For birth rates $b_i$ and death rates $d_i$, a reversible measure satisfiesThus ${\pi }_{i+1}/{\pi }_i=b_i/d_{i+1}$, and it is an invariant distribution when the resulting measure is summable and the chain is nonexplosive.

An Equilibrium distribution of a continuous-time Markov chain is a probability row vector $\pi$ invariant under its transition semigroup:It satisfies $\pi Q=0$ when the generator sums are defined. An invariant law is a property of the entire evolution, so formal generator equations must be interpreted with the process's nonexplosion and domain conditions.

For a finite irreducible chain, the equilibrium distribution is unique and every transition row converges to it. For a general finite chain, the limiting row from state $i$ is a mixture of the stationary distributions ${\pi }^C$ of its closed irreducible classes:The chain eventually enters one of these classes and then mixes within it. If every closed class is a single absorbing state, this reduces to absorption probabilities at those states. Merely having some absorbing states does not exclude other closed classes.

Holding times explain the link to an embedded jump chain. For a finite irreducible chain with strictly positive exit rates $q_i$ and jump-chain Stationary distribution $\alpha$, the time-stationary distribution isVisit frequency is multiplied by mean time $1/q_i$ per visit. In the countable setting, these time weights need a finite positive normalizer and a nonexplosive process before they define an equilibrium probability. For an alternating two-state jump chain, $\alpha =(1/2,1/2)$ and rates $a,b$ give $\pi =(b/(a+b),a/(a+b))$: slower holding rates increase time occupancy.

Consider states $C_i,W_i$ for integer levels $i\ge 0$ and strictly positive rates $\lambda ,\mu ,\alpha ,\beta$. From $C_i$ the chain moves to $C_{i+1}$ at rate $\lambda$ and to $W_i$ at rate $\alpha$. From $W_i$, $i\ge 1$, it moves to $W_{i-1}$ at rate $\mu$ or to $C_i$ at rate $\beta$. At the reflecting boundary $W_0$ there is only the switch to $C_0$; no negative level is allowed. All rates are bounded, so the chain is nonexplosive.

Let $h$ be the probability of hitting $W_0$, with $h_{W_0}=1$. First-step equations arewith the first equation also valid at $i=0$. Put $\vartheta =\mu (\lambda +\alpha )/[\lambda (\mu +\beta )]$. If $\mu \alpha <\lambda \beta$, then $\vartheta <1$ andSubstitution verifies both equations and the boundary. Eliminating the $C$ variables gives a second-order recurrence with modes $1,{\vartheta }^i$. The displayed decaying solution is the minimal nonnegative harmonic solution and hence the Hitting probability.

When $\mu \alpha \ge \lambda \beta$, the growing mode is excluded by boundedness; at equality the second mode is linear and is likewise excluded. The only bounded solution with the boundary value is one, proving almost-sure hitting. Since the specified reflected chain is irreducible, these cases classify it as recurrent or transient respectively. Specifying the boundary rule is essential for that recurrence statement; making $W_0$ absorbing would be a different chain.

Add independent rate-$\mu$ resets to zero to a linear birth process with immigration. The resulting irreducible continuous-time Markov chain is nonexplosive by domination of births and finiteness of Poisson reset counts. When $\mu =\lambda$, an excursion from one reaches $n$ with probability $2/(n+1)$, so its jump chain is recurrent with infinite expected jump count. The elapsed return-cycle time instead has meanThus the continuous-time chain is positive recurrent while its jump chain is null recurrent. Its stationary probabilities are ${\pi }_0=1/2$ and ${\pi }_n=1/[(n+1)(n+2)]$ for $n\ge 1$.

A Markov jump process is a nonexplosive, piecewise-constant continuous-time Markov process. Transition rates $q(x,y)$ give exponential holding time of rate ${\sum }_{y\ne x}q(x,y)$ and next-state probabilities proportional to these rates. Being Markov relative to a specified filtration requires the conditional future law given that filtration to depend only on the present state.

// Target: probability-and-statistics.bigb

A pure birth process is nonexplosive exactly whenIts explosion time is the sum of independent exponential holding times. If the displayed series converges, that sum has finite expectation. If it diverges, the Laplace transform of the sum isforcing the sum to be infinite almost surely.

A continuous-time Markov chain with finite holding rates is nonexplosive when its jump times have no finite accumulation almost surely. It then remains in its specified state space at every finite time, rather than being lost to a cemetery state after infinitely many jumps. Bounded rates are sufficient for nonexplosion, but finite rates at individual states alone are not.

A continuous-time Markov chain is nonexplosive when it makes only finitely many jumps in every finite time interval, almost surely. Uniformly bounded total jump rates suffice. Another useful criterion bounds the states visited over each finite interval by a random finite set on which rates are bounded.

Only routes with $n_r>0$ need a per-flow rate: absent flows contribute no utility and have no completion events. Interpret the proportional-change sum and the logarithmic objective on these active routes. With positive capacities and finitely many nonempty routes, a strictly positive feasible active-rate vector exists. Define its weighted logarithmic utility by $U(x)={\sum }_{r:n_r>0}{n}_r\log x_r$, with value $-\infty$ if an active rate is zero.

For positive feasible $x,y$, the elementary concavity inequality $\log u\le u-1$ givesThe last sum is nonpositive by proportional fairness. Feasible competitors with a zero active rate have utility $-\infty$, so they cannot improve it either. Thus a proportionally fair allocation maximizes weighted logarithmic utility. Conversely, if $x$ maximizes this utility, the feasible line segment $x+\theta (y-x)$ has nonpositive right derivative at $\theta =0$, giving precisely ${\sum }_r{n}_r(y_r-x_r)/x_r\le 0$. This also covers a competitor with zero coordinates, since $x+\theta (y-x)$ stays positive for sufficiently small $\theta$. The strictly concave function $U$ makes the active-rate optimizer unique.

For the linear flow network, put $M={\sum }_{i=1}^I{n}_i$ and $N=n_0+M$. If $n_i>0$ and resource $i$ is not saturated, its local per-flow rate could be increased without affecting any other resource. This would strictly increase weighted logarithmic utility. HenceIf $n_0>0$ and $M>0$, substitute $x_i=(1-n_0{x}_0)/n_i$ on all active local routes into the utility. The terms depending on $x_0$ areIts derivative vanishes when $1/x_0=M/(1-n_0{x}_0)$, so $x_0=1/N$. Its second derivative is negative, giving the unique maximizer. If $M=0<n_0$, utility increases up to the capacity endpoint $x_0=1/n_0=1/N$. If $n_0=0$, each active local route uses its entire local resource. Altogether the proportionally fair allocation on a linear flow network isAbsent-route rates may be set to zero, and departures are zero at the empty state $N=0$. Notice that these are per-flow rates: the aggregate through service is $n_0/N$, while each active local route gets aggregate service $M/N$.

In the document-transfer flow-level network model, let $e_r$ be the unit vector for route $r$. Independent Poisson processes of flow arrivals give transitions $n\to n+e_r$ at rate ${\nu }_r$. An active document on route $r$ receives service at rate $x_r(n)$ and has an exponential distribution of size with parameter ${\mu }_r$. By the memoryless property, its instantaneous completion rate is ${\mu }_r{x}_r(n)$. Summing over the active documents gives the continuous-time Markov chain intensitiesThus through-route departures have rate ${\mu }_0{n}_0/N$, and active local departures have rate ${\mu }_{i}M/N$. All other off-diagonal intensities vanish. The diagonal intensity is the negative of the sum of the outgoing intensities. Each route's aggregate service is at most one, so the total jump rate is bounded by ${\sum }_r({\nu }_r+{\mu }_r)$; the process is a nonexplosive continuous-time Markov chain.

Set ${\rho }_r={\nu }_r/{\mu }_r$, assume ${\mu }_r>0$, and define the balance function of a flow-level network byThe adjacent-state ratios of this binomial coefficient areThey equal the aggregate service rates just obtained. Therefore, with $w(n)=H(n){\prod }_r{\rho }_r^{n_r}$,These are the detailed balance for a continuous-time Markov chain equations on every adjacent pair. Summing incoming and outgoing pairs proves stationarity whenever the weights can be normalized. Hence the stationary law of a linear flow network isThe coefficient here is a binomial coefficient, not merely the total number of flows. Both it and the denominator in $x_0=1/N$ are visible in the original PDF but missing from the converted TeX.

We can also determine exactly when the normalizing constant is finite. For $0\le {\rho }_0<1$, summing first over $n_0$ with local counts fixed and applying the negative binomial series givesThe remaining sums are independent geometric series. Thusprovided ${\rho }_0+{\rho }_i<1$ for every $i$. These inequalities are also necessary: if ${\rho }_0\ge 1$, the states with every local count zero already give a divergent sum; otherwise any local ratio at least one makes its geometric series diverge. Thus the stationary formula requires strict offered load below capacity at every resource. With positive arrival rates, the process is irreducible, and these normalizable detailed balance weights give its unique stationary distribution. Outside these inequalities, the displayed weights are not a probability distribution. Zero arrival rates can be handled on the reachable class by setting the corresponding population to zero.

Fix $M$. Each time the chain is in $\{1,\dots ,M\}$ immediately before a jump, its conditional chance to jump to zero is at least the $\delta$ from part (b). Count all these opportunities, including returns to the same state after an excursion; the $T_m$ in part (b) need not count those returns. By the Strong Markov property at successive such visits, the probability of surviving $r$ opportunities without absorption is at most $(1-\delta {)}^r$. Hence infinitely many such opportunities without hitting zero have probability zero.

A positive state cannot be occupied forever: its finite positive holding rate gives an almost surely finite holding time. For a nonexplosive continuous-time Markov chain, visits at arbitrarily late times to the finite set therefore require infinitely many of the counted opportunities. On the event of no absorption, the chain consequently eventually stays above $M$, almost surely. Intersect this probability-one assertion over integers $M$. The result is $X_t\to \infty$. If zero is ever reached it remains occupied forever, and for initial state zero this is immediate. Thus $\boxed{{\mathbb{P}}_i(E\cup F)=1}$ for every initial state.

This uses the usual assumption implicit in an $I$-valued chain defined for all $t\ge 0$: no explosion with loss to a cemetery state. The rate conditions alone do not ensure it. For example rates $q_{i,i+1}=i^3$, $q_{i0}=1$ allow an infinite upward path with probability ${\prod }_i{i}^3/(i^3+1)>0$ and a finite total holding time, since ${\sum }_{i}1/(i^3+1)<\infty$. The minimal chain then explodes; with a cemetery state the printed dichotomy would require an additional explosion alternative.

A product-Poisson stationary distribution is suggested by balancing the three reaction fluxes around the directed cycle of reaction complexes. If their mean counts are $a,b$, this complex balance condition iswhich gives $a=4$ and $b=1/2$. This suggests the joint probability mass functionIt is normalized because it is the product of two Poisson distributions. We verify it directly in the chemical master equation, rather than assuming a product-Poisson stationary distribution theorem. The three incoming terms satisfyWith negative-index values set to zero, these identities also hold on the axes. Their sum equals $(2+n/2+nm)\pi (n,m)$, precisely the outgoing probability flux. The stationary average total exit rate is $2+{\mathbb{E}}_{\pi }n/2+{\mathbb{E}}_{\pi }(nm)=6<\infty$, so these stationary balance equations do define an invariant distribution of a continuous-time Markov chain for the nonexplosive model verified below. This is complex balance; it does not assert pairwise detailed balance for the irreversible chemical reactions.

To identify this invariant distribution with the limiting probability distribution in the question, first check that the model is a nonexplosive continuous-time Markov chain. On any finite time interval, $A_1$ is bounded by its initial count plus the finite number of arrivals in a rate-$2$ Poisson process. Conditional on that bound, the rate of producing $A_2$ is bounded, so only finitely many such productions occur. The number of joint removals cannot exceed the available molecules. Thus only finitely many chemical reactions occur in finite time.

The continuous-time Markov chain is also an irreducible Markov chain. From a Markov chain state with both counts positive, repeated joint removals reduce both. On the $A_2=0$ axis, producing one $A_2$ and then removing a pair reduces $A_1$ by one. On the $A_1=0$ axis, adding one $A_1$ and then removing a pair reduces $A_2$ by one. Each prescribed finite sequence has positive probability, so every Markov chain state reaches $(0,0)$. Conversely, arrivals followed by productions reach any $(n,m)$ with $n>0$; to reach $(0,m)$, reach $(1,m+1)$ and remove one pair. The continuous-time Markov chain is therefore positive recurrent, and the continuous-time Markov chain convergence theorem gives $\varphi =\pi$ from the empty initial condition.

The two Poisson factors happen to have cancelling numerical coefficients at $(2,2)$:

The Yule process is the pure birth process with rate $n\lambda$ in population state $n$. Starting from one individual, it is nonexplosive because the independent holding times have divergent total almost surely.