Past exam of the mathematics course of the University of Cambridge / 2009 / ii / Paper 1 / 27J / b / Solution

The chain is not an irreducible Markov chain, since zero is absorbing and cannot reach a positive state. Write $q_i=-q_{ii}={\sum }_{j\ne i}{q}_{ij}$. These are finite positive holding rates for the usual continuous-time Markov chain. Its first holding time in state $i\ge 1$ is exponential with rate $q_i$, and its jump destination has probabilities $q_{ij}/q_i$. ThereforeThe recursively defined $T_m$ are stopping times: after $T_m$, wait until the path first reaches a different state in the finite set. Set all later times to infinity if a previous time is infinite. On $A_m$, the Strong Markov property applies at $T_m$. Its next jump goes to zero with probability at least $\delta$; if it does, $T_{m+1}=\infty$. Other jumps may also prevent another qualifying visit. ThusStarting with $A_0$ certain gives $\boxed{{\mathbb{P}}_i(A_m)\le (1-\delta {)}^m}$.