Birth-death recurrence classification Created 2026-09-24 Updated 2026-11-05
Consider a birth-death process on with constant birth rate , constant death rate at every positive state, and no deaths from zero. The reflecting boundary is essential: the result changes if zero is an absorbing state. Detailed balance would giveWhen , these weights normalize to a Geometric distribution, so the irreducible chain is positive recurrent. When , the embedded jump chain is the symmetric reflected random walk: it returns to zero almost surely, but its constant equilibrium weights have infinite sum, so the chain is null recurrent. When , its upward-biased embedded random walk has probability of hitting zero from state one, making the chain transient. The holding rates are bounded, so nonexplosion creates no additional qualification here.
Continuous-time Markov chain Created 2026-09-24 Updated 2026-11-05
A Continuous-time Markov chain on a finite or countable state space is specified by transition rates for , with finite total exit rate . At a state with , it waits an exponential time of rate and then moves to with probability . A state with is absorbing: it has infinite holding time rather than a zero-rate exponential waiting law.
Its transition-rate matrix is , with off-diagonal entries and diagonal entries , so each row sums to zero. The rates describe first-order transition probabilities: for . On a finite state space, the transition semigroup is and explosion is impossible. On a countably infinite state space, finite individual exit rates do not alone exclude infinitely many jumps in finite time; nonexplosion must be checked separately.
For two states with rates and , the mean holding times are and . Detailed balance gives Stationary distribution , andThis starts at one and relaxes to the stationary mass of state zero. Changing holding rates can preserve an embedded jump chain while changing the proportions of elapsed time spent in its states.
Equilibrium distribution of a continuous-time Markov chain Created 2026-10-14 Updated 2026-11-05
An Equilibrium distribution of a continuous-time Markov chain is a probability row vector invariant under its transition semigroup:It satisfies when the generator sums are defined. An invariant law is a property of the entire evolution, so formal generator equations must be interpreted with the process's nonexplosion and domain conditions.
For a finite irreducible chain, the equilibrium distribution is unique and every transition row converges to it. For a general finite chain, the limiting row from state is a mixture of the stationary distributions of its closed irreducible classes:The chain eventually enters one of these classes and then mixes within it. If every closed class is a single absorbing state, this reduces to absorption probabilities at those states. Merely having some absorbing states does not exclude other closed classes.
Holding times explain the link to an embedded jump chain. For a finite irreducible chain with strictly positive exit rates and jump-chain Stationary distribution , the time-stationary distribution isVisit frequency is multiplied by mean time per visit. In the countable setting, these time weights need a finite positive normalizer and a nonexplosive process before they define an equilibrium probability. For an alternating two-state jump chain, and rates give : slower holding rates increase time occupancy.
Explosion of an upward-biased walk with geometrically increasing rates Created 2026-09-24 Updated 2026-11-05
Suppose a transient nearest-neighbour jump chain on has uniformly bounded expected visits to each state and the holding rate at grows geometrically. Thenso the continuous-time chain explodes almost surely.
Fast holding rates create continuous-time positive recurrence Created 2026-09-24 Updated 2026-11-05
A recurrent jump chain with infinite invariant measure can yield a positive-recurrent continuous-time chain when .
Invariant distribution of an explosive chain Created 2026-09-24 Updated 2026-11-05
For an explosive chain, a normalized solution of the formal balance equations need not be an invariant probability for the minimal transition semigroup. Explosion sends Probability mass to a cemetery state, and the generator equations on the original states do not include that boundary loss.
A concrete example uses the reflected jump chain on which moves from zero to one, and from moves up with probability and down with probability . Give state holding rate . The embedded chain has invariant weights , for . Dividing by the holding rates and normalizing yieldsThese weights even satisfy formal Detailed balance: , and for both opposing edge flows equal . Thus holds at each state.
Nevertheless the process explodes. Starting from zero, the embedded walk has expected visit count two at zero and three at every positive level. The zero-state return probability is ; at a positive state it is , giving those geometric visit means. Therefore the expected total elapsed holding time isThe lifetime is finite almost surely. The same bounded expected-visit argument works from every initial state, so the killed minimal semigroup loses all its mass as time tends to infinity. The displayed is consequently a formal generator-balance solution, not a stationary probability of that killed evolution. A conservative continuation after explosion would require an extra boundary rule.
Invariant-measure transfer between a jump chain and a CTMC Created 2026-09-24 Updated 2026-11-05
For strictly positive finite holding rates , a measure satisfies exactly when is invariant for the jump chain. Thus when normalization is possible.
Irreducibility and recurrence pass between a CTMC and its jump chain Created 2026-09-29 Updated 2026-11-05
For a continuous-time Markov chain with strictly positive finite holding rates whose jump times tend to infinity, the chain and its jump chain have the same directed paths of positive-probability jumps. They are therefore irreducible simultaneously, and a state is recurrent for one exactly when it is recurrent for the other: the event of returning after departure depends only on the sequence of visited states.
Jump chain Created 2026-09-24 Updated 2026-11-05
The jump chain records the successive states visited by a Continuous-time Markov chain while discarding its holding times.
Linear birth process with catastrophes Created 2026-10-14 Updated 2026-11-05
Add independent rate- resets to zero to a linear birth process with immigration. The resulting irreducible continuous-time Markov chain is nonexplosive by domination of births and finiteness of Poisson reset counts. When , an excursion from one reaches with probability , so its jump chain is recurrent with infinite expected jump count. The elapsed return-cycle time instead has meanThus the continuous-time chain is positive recurrent while its jump chain is null recurrent. Its stationary probabilities are and for .
Nonexplosion from recurrent visits to a slow state Created 2026-09-24 Updated 2026-11-05
If the jump chain visits a state of finite positive holding rate infinitely often, the sum of its independent positive holding times at that state diverges almost surely, preventing explosion.
Occupation time at an interior state of a variable-rate symmetric walk Created 2026-10-14 Updated 2026-11-05
For a continuous-time Markov chain with absorbing endpoints and rates , the jump chain is a simple symmetric random walk. On leaving , absorption before returning has probability , by gambler's ruin on each side. The geometric return count before absorption and independent rate- holding times give . The rates at other states affect elapsed excursion times but not their return probabilities.
Past exam of the mathematics course of the University of Cambridge 2003 iii Paper 33 6 Solution Created 2026-10-03 Updated 2026-10-14
Let be a Markov jump process on integer populations, with rateswhen , where . Make both axes absorbing, so the fight ends when one population reaches zero. Start at , with ; when is integral this is the printed initial condition exactly. The harmless rounding convention makes the model defined for every integer .
Use the embedded jump chain rather than physical time. Its probabilities of the two losses are respectively and , independently of . It reaches an axis after at most steps; hold it constant thereafter. PutBefore absorption its two increments are and , soAfter absorption set . Consequentlyis a discrete-time martingale. All populations are at most , so each martingale difference is bounded in absolute value by . The total drift up to step is at most . This is the squared-population near-invariant for stochastic attrition.
The two-sided Azuma-Hoeffding inequality states that a martingale with differences bounded by satisfies . Its exponential estimate follows by applying the conditional Hoeffding lemma to each centered difference, iterating, and optimizing the exponential Markov bound. With and it gives
Let , and let be the final proportion. If the first population survives, then the second is zero and . Hence impliesIf the first population is extinguished, and . This can satisfy the indicated deviation event only when , and then . Thus the same implication holds in both cases.
The initial rounding error obeys . Combining it with the drift bound shows that the deviation event forcesfor all sufficiently large . Set in the preceding concentration inequality. We concludefor sufficiently large , and thereforeThe symmetric stochastic attrition process thus has exponential concentration of terminal stochastic attrition survivors. The deterministic equations conserve and predict the same surviving fraction, but the martingale argument supplies the stronger exponential probability estimate required here.
Past exam of the mathematics course of the University of Cambridge 2008 ii Paper 1 26I iv Solution Created 2026-09-24 Updated 2026-10-14
The jump chain records the successive states visited at actual jumps of the continuous-time Markov chain. Put . For an irreducible chain with at least two states, all , and its transition probabilities are for , with . The holding time in state is exponential with mean .
If , set and . ThenConversely, implies that , normalized to sum one, satisfies . ThereforeJump observations overrepresent states with short holding times. A one-state chain has no actual jumps; its trivial equilibrium can be handled separately, without using this division by .
Past exam of the mathematics course of the University of Cambridge 2008 ii Paper 1 26I vi Solution Created 2026-09-24 Updated 2026-10-14
Write for , with a common positive constant , and put . The infinitesimal generator matrix is symmetric because the graph is undirected. Thus the uniform distribution satisfies detailed balance, and the continuous-time random walk is reversible when started uniformly.
Its jump chain chooses a link uniformly from those incident to its current vertex:The flux is symmetric, so this random walk on a multigraph is also reversible. The continuous-time equilibrium is uniform because higher-degree vertices have shorter holding times; at jump times the equilibrium is degree-weighted. If the intended proportionality constant depends on the starting vertex, , reversibility still holds with , and the same jump chain results.
Past exam of the mathematics course of the University of Cambridge 2008 ii Paper 1 26I v Solution Created 2026-09-24 Updated 2026-10-14
Use the equilibrium correspondence from the preceding part. For ,Consequently the detailed balance equations for the jump chain hold exactly when those for the continuous-time Markov chain hold. The reversibility characterizations already given prove the equivalence. Both chains here are started in their respective stationary distributions, which generally differ.
Past exam of the mathematics course of the University of Cambridge 2010 ii Paper 1 27I e Solution Created 2026-09-24 Updated 2026-10-14
At a positive-rate state the jump chain moves to either other state with probability , independently of the holding rate. At a zero-rate state there is no subsequent jump; extend the chain as permanently absorbed there, giving an identity row. This convention is needed to speak of all powers of its transition matrix in the absorbing cases.
When all rates are positive,Its unique equilibrium is uniform and its powers tend to the matrix with all rows . This differs from the holding-time-weighted equilibrium of the continuous-time chain unless the rates coincide.
With one zero-rate state the unique jump-chain equilibrium is its point mass and the limit has all rows that point mass. With two zero-rate states the equilibria are all mixtures on them, and the limiting rows are exactly the absorbing rows and half-half transient row found in part (d).
Past exam of the mathematics course of the University of Cambridge 2010 ii Paper 4 26I a ii Solution 2026-10-14
A state is recurrent if, after departure, its return probability is one. For an irreducible nonexplosive continuous-time Markov chain, this is equivalent to recurrence of its jump chain and is shared by all states.
Past exam of the mathematics course of the University of Cambridge 2010 ii Paper 4 26I a i Solution 2026-10-14
A state of an irreducible continuous-time Markov chain is transient if, after departure, its return probability is less than one. Equivalently its jump chain visits that state only finitely often almost surely. This is a class property.
Past exam of the mathematics course of the University of Cambridge 2010 ii Paper 4 26I a iv Solution 2026-10-14
A recurrent state is null recurrent if its expected elapsed return-cycle time is infinite. Recurrence itself agrees with that of the jump chain, but positive recurrence need not agree, because large states can have very short holding times.