Markov jump process

A Markov jump process is a nonexplosive, piecewise-constant continuous-time Markov process. Transition rates $q(x,y)$ give exponential holding time of rate ${\sum }_{y\ne x}q(x,y)$ and next-state probabilities proportional to these rates. Being Markov relative to a specified filtration requires the conditional future law given that filtration to depend only on the present state.

// Target: probability-and-statistics.bigb

This finite-state Markov jump process models two opposing populations, each suffering losses at a rate proportional to the surviving opposing population. Both coordinate axes are absorbing. Its embedded jump chain removes one individual at each step, with probabilities $b/(a+b)$ and $a/(a+b)$. The corresponding deterministic equations $\dot{a}=-\kappa b$, $\dot{b}=-\kappa a$ conserve $a^2-b^2$; finite stochastic populations satisfy a near-invariant instead.

In the jump chain of the symmetric stochastic attrition process, the two possible increments of $D$ are $-2A+1$ and $2B-1$. Their mean is the displayed quantity, bounded by one in absolute value. Subtracting these conditional means produces a martingale. If both initial populations are at most $N$, it has at most $2N$ nonzero increments, each bounded by $4N$. Thus Azuma-Hoeffding inequality controls changes of order $N^2$ with exponentially small probability of order $e^{-cN}$.

Start the symmetric stochastic attrition process at $(N,\lfloor \alpha N\rfloor )$, with $0<\alpha <1$, and let $X^N$ be the final first-population size divided by $N$. The bound holds for all sufficiently large $N$, depending on $\alpha ,\delta >0$. At absorption, one coordinate is zero. If the first survives, the terminal squared difference is $N^2(X^N{)}^2$; if it is extinguished, that difference is nonpositive. In either case the indicated deviation forces an error of at least $\sqrt{1-{\alpha }^2}\delta N^2$ from the limiting squared invariant. The initial rounding error and total conditional drift are only $O(N)$. Apply Azuma-Hoeffding inequality to the centered squared-population near-invariant for stochastic attrition, using its $2N$ increments bounded by $4N$, to obtain the displayed exponent.

Let $T$ be a rate-one exponential random variable independent of a uniform random variable $U$ on $(0,1)$. In its natural filtration, this Markov jump process jumps once from zero to its mark and then stays there. Its destination rate kernel is $q(x,dy)=1_{\{x=0\}}{1}_{(0,1)}(y)dy$. For a Borel mark set $B$, the martingale $1_{\{T\le t,U\in B\}}-|B|(t\wedge T)$ follows directly from the memoryless residual lifetime. Information about the future mark or lifetime must not be inserted into the initial filtration.

A kernel specifying the jump rate from state $x$ for each displacement $z$. With the displacement convention the infinitesimal generator is $L\phi (x)=\int (\phi (x+z)-\phi (x))K(x,dz)$. The destination convention instead records a rate measure on the new state, translating each displacement by $x$. For finite total rate, that total is the exponential holding rate and the normalized measure is the next-jump distribution.

With the displacement convention for a Lévy kernel of a Markov jump process, suppose its total rate is bounded, its first moment is zero and its second moment $v$ is bounded by $V$. The bounded rate prevents explosion. The compensated displacement integral is a square-integrable martingale with $\mathbb{E}{X}_t^2\le Vt$. The displayed process is the sum of $2\int X_{s-}d{X}_s$ and the compensated sum of squared jumps. The first term is square integrable since $4\mathbb{E}{\int }_0^t{X}_{s-}^{2}v(X_{s-})ds\le 2V^2{t}^2$; the second is integrable because its expected total jump-square sum is at most $Vt$. Consequently the square compensation is a true martingale, without a fourth-moment assumption.

For a Markov jump process with off-diagonal rate kernel $q(x,dy)$, the jump measure records the jump time and the destination state. Relative to a filtration for which the process has the Markov jump property, its predictable compensator is the displayed rate measure, up to explosion. The corresponding generator is $Qf(x)=\int [f(y)-f(x)]q(x,dy)$. An enlarged filtration revealing future jumps need not preserve this compensator, so the filtration condition is essential.

The predictable random measure $\nu$ matching the expected integrals of the jump measure $N$ against every nonnegative predictable integrand. For displacement marks of a Markov jump process, $\nu (ds,dy)=K(X_{s-},dy)ds$. Multiplying the integrand by an event known before an interval proves that the compensated integral has conditionally centered increments; expected absolute integrability makes it a true martingale. Squared-integrand integrability gives the square-integrable compensation isometry.

A predictable mark-dependent integrand $H(t,y)$ gives a true martingale ${\int }_0^t\int H(s,y)(\mu -\nu )(ds,dy)$ whenever $\mathbb{E}{\int }_0^t\int |H|d\nu <\infty$ on every finite horizon. Local versions follow when this expected integrability holds after localization. The compensator identity applied to predictable indicators from the past proves that every increment has conditional expectation zero. Choosing $H(s,y)=f(y)-f(X_{s-})$ gives the generator martingale $f(X_t)-f(X_0)-{\int }_0^{t}Qf(X_s)ds$ under the same local integrability conditions.

With scale $N$, such a process has finitely many possible jump vectors $\ell /N$ occurring at rates $N{\beta }_{\ell }(x)$ depending on the current scaled state. Its drift is ${\sum }_{\ell }\ell {\beta }_{\ell }(x)$, while the predictable covariance rate of its martingale fluctuations is $N^{-1}{\sum }_{\ell }\ell {\ell }^T{\beta }_{\ell }(x)$.

// Target: probability-and-statistics.bigb

In a population of $N$ playing rock paper scissors, let losers copy winners. With pair-selection events of rate $(N-1)/2$, the proportion process has jumps $(e_1-e_3)/N$, $(e_2-e_1)/N$, $(e_3-e_2)/N$ at rates $N{x}_1{x}_3$, $N{x}_2{x}_1$, $N{x}_3{x}_2$. Its fluid limit obeys ${\dot{x}}_i=x_i(x_{i-1}-x_{i+1})$, with indices interpreted cyclically. The product of numbers $x_1{x}_2{x}_3$ is conserved in this ordinary differential equation, whereas the finite chain eventually reaches a monochromatic absorbing state.

For the cyclic imitation chain, its infinitesimal generator satisfies ${\mathcal{L}}_N(x_1{x}_2{x}_3)=-3x_1{x}_2{x}_3/N$. Therefore $\mathbb{E}[X_1^N(t)X_2^N(t)X_3^N(t)]=e^{-3t/N}\mathbb{E}[X_1^N(0)X_2^N(0)X_3^N(0)]$. While all three types remain, the product of numbers is at least $(N-2)/N^3$ for $N\ge 3$, so this identity proves that one type is lost almost surely. The remaining pair has a fixed winner, which eventually occupies the whole population. The conserved product in the deterministic fluid limit therefore describes fixed finite horizons, not the chain's infinite-time behavior.

With jumps $\ell /N$ at rates $N{\beta }_{\ell }(z)$, compensate each jump count by its integrated rate. The drift is $F(z)={\sum }_{\ell }\ell {\beta }_{\ell }(z)$ and the predictable quadratic variation matrix of the martingale term is $N^{-1}{\int }_0^t{\sum }_{\ell }\ell {\ell }^{\mathsf{T}}{\beta }_{\ell }(Z_s^N)ds$. On a compact region with bounded rates this bracket is of order $1/N$. A Lipschitz drift and the Gronwall inequality transfer martingale concentration to the fluid limit.