Past exam of the mathematics course of the University of Cambridge / 2009 / ii / Paper 1 / 27J (Applied Probability)

For the natural filtration ${\mathcal{F}}_t=\sigma (X_s:s\le t)$, a stopping time $T$ takes values in $[0,\infty ]$ and satisfies $\{T\le t\}\in {\mathcal{F}}_t$ for every $t$. Its occurrence is decidable from information available by that time.

The Strong Markov property says that on $T<\infty$, conditionally on $X_T$, the post-$T$ process is independent of ${\mathcal{F}}_T$ and has the law of the original continuous-time Markov chain started at $X_T$. In particular ${\mathbb{E}}_i[f(X_{T+s})\mid {\mathcal{F}}_T]=P_{s}f(X_T)$ on $T<\infty$. The chain is an irreducible Markov chain if every state communicates with every other: for any $i,j$ some time has ${\mathbb{P}}_i(X_t=j)>0$, equivalently for finite rates there is a path of positive off-diagonal jump rates between them.

The chain is not an irreducible Markov chain, since zero is absorbing and cannot reach a positive state. Write $q_i=-q_{ii}={\sum }_{j\ne i}{q}_{ij}$. These are finite positive holding rates for the usual continuous-time Markov chain. Its first holding time in state $i\ge 1$ is exponential with rate $q_i$, and its jump destination has probabilities $q_{ij}/q_i$. ThereforeThe recursively defined $T_m$ are stopping times: after $T_m$, wait until the path first reaches a different state in the finite set. Set all later times to infinity if a previous time is infinite. On $A_m$, the Strong Markov property applies at $T_m$. Its next jump goes to zero with probability at least $\delta$; if it does, $T_{m+1}=\infty$. Other jumps may also prevent another qualifying visit. ThusStarting with $A_0$ certain gives $\boxed{{\mathbb{P}}_i(A_m)\le (1-\delta {)}^m}$.

Fix $M$. Each time the chain is in $\{1,\dots ,M\}$ immediately before a jump, its conditional chance to jump to zero is at least the $\delta$ from part (b). Count all these opportunities, including returns to the same state after an excursion; the $T_m$ in part (b) need not count those returns. By the Strong Markov property at successive such visits, the probability of surviving $r$ opportunities without absorption is at most $(1-\delta {)}^r$. Hence infinitely many such opportunities without hitting zero have probability zero.

A positive state cannot be occupied forever: its finite positive holding rate gives an almost surely finite holding time. For a nonexplosive continuous-time Markov chain, visits at arbitrarily late times to the finite set therefore require infinitely many of the counted opportunities. On the event of no absorption, the chain consequently eventually stays above $M$, almost surely. Intersect this probability-one assertion over integers $M$. The result is $X_t\to \infty$. If zero is ever reached it remains occupied forever, and for initial state zero this is immediate. Thus $\boxed{{\mathbb{P}}_i(E\cup F)=1}$ for every initial state.

This uses the usual assumption implicit in an $I$-valued chain defined for all $t\ge 0$: no explosion with loss to a cemetery state. The rate conditions alone do not ensure it. For example rates $q_{i,i+1}=i^3$, $q_{i0}=1$ allow an infinite upward path with probability ${\prod }_i{i}^3/(i^3+1)>0$ and a finite total holding time, since ${\sum }_{i}1/(i^3+1)<\infty$. The minimal chain then explodes; with a cemetery state the printed dichotomy would require an additional explosion alternative.