Past exam of the mathematics course of the University of Cambridge / 2010 / ii / Paper 4 / 26I (Applied Probability)

An irreducible nonexplosive continuous-time Markov chain has an equilibrium probability distribution exactly when it is positive recurrent. The distribution is unique, and gives the long-run fraction of elapsed time in each state. A null recurrent chain has no equilibrium probability distribution. The finite one-state absorbing case is trivially stationary; the return-cycle descriptions concern nontrivial irreducible classes.

A state of an irreducible continuous-time Markov chain is transient if, after departure, its return probability is less than one. Equivalently its jump chain visits that state only finitely often almost surely. This is a class property.

A state is recurrent if, after departure, its return probability is one. For an irreducible nonexplosive continuous-time Markov chain, this is equivalent to recurrence of its jump chain and is shared by all states.

A recurrent state is positive recurrent if its expected elapsed return-cycle time is finite. The cycle starts on entering the state and ends at its next entrance after a departure; it includes the initial holding time. This convention avoids counting the immediate return caused by a right-continuous path remaining at its initial state.

A recurrent state is null recurrent if its expected elapsed return-cycle time is infinite. Recurrence itself agrees with that of the jump chain, but positive recurrence need not agree, because large states can have very short holding times.

For this linear birth process with catastrophes, the off-diagonal entries of the Q-matrix arewith all other off-diagonal entries zero. The diagonal entries are $q_{00}=-\lambda$ and $q_{nn}=-[\lambda (n+1)+\mu ]$ for $n\ge 1$. At zero there is no separate catastrophe transition, because it would not change the state.

The chain is irreducible: every positive state can jump to zero, and from zero any finite sequence of births has positive probability. To prove nonexplosion, couple it with the linear birth process with immigration having birth rate $\lambda (n+1)$ and no catastrophes. Its mean population is finite at each finite time, obtained from $m^{\prime }=\lambda (m+1)$, or its exponential holding-time sum diverges since ${\sum }_{n}1/[\lambda (n+1)]=\infty$. Couple births monotonically and implement catastrophes with an independent rate-$\mu$ Poisson clock, resetting the smaller population. On a finite time interval the dominating pure-birth process has finitely many births, and the catastrophe clock finitely many rings. Thus the original chain is nonexplosive.

For its jump chain, the transitions are

When $\mu =\lambda$, an excursion after the jump from zero to one visits $n$ before its reset with probabilityThis tends to zero, so a catastrophe eventually occurs with probability one. Both the original chain and its jump chain are recurrent.

The expected number of jumps in a return cycle isThus the jump chain is null recurrent. But its holding time at $n\ge 1$ has mean $1/[\lambda (n+2)]$, and the initial holding time at zero has mean $1/\lambda$. The expected elapsed cycle time isThe continuous-time Markov chain is therefore positive recurrent. Its equilibrium distribution, computed as mean time in a state per mean cycle time, isThe entries sum to one by telescoping. The contrast is explained by progressively shorter holding times at the large states, not by a change in return probabilities.