Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2009/ii/paper-1/27j/b/solution

The chain is not an irreducible Markov chain, since zero is absorbing and cannot reach a positive state. Write . These are finite positive holding rates for the usual continuous-time Markov chain. Its first holding time in state is exponential with rate , and its jump destination has probabilities . Therefore
The recursively defined are stopping times: after , wait until the path first reaches a different state in the finite set. Set all later times to infinity if a previous time is infinite. On , the Strong Markov property applies at . Its next jump goes to zero with probability at least ; if it does, . Other jumps may also prevent another qualifying visit. Thus
Starting with certain gives .

New to topics? Read the docs here!