Past exam of the mathematics course of the University of Cambridge / 2020 / ib / Paper 1 / 20H (Markov Chains)

A random time $\tau$ is a stopping time for the natural filtration $({\mathcal{F}}_n)$ of a Markov chain when $\{\tau \le n\}\in {\mathcal{F}}_n$ for every $n$. The Strong Markov property says that, conditionally on $\tau <\infty$ and $X_{\tau }=i$, the process after $\tau$ is a fresh copy of the chain started from $i$, independent of the pre-$\tau$ history.

Before capture define the half-separationFor $D_n=k\ge 2$, independence of the two moves gives the transition probabilitiesIgnore the holding steps. The resulting embedded chain is a biased random walk that moves left with probability $q$ and right with probability $1-q$. Its expected number of moves needed to descend one level is $1/(2q-1)$; this follows either from first-step analysis or from its drift $1-2q$. A non-holding move occurs with probability $1/2$ at each time, so its mean waiting time is two. Hence, for every $m\ge 2$, the expected time to go from separation $2m$ to $2(m-1)$ is

Let $T_k$ be the expected capture time starting from $D_0=k$, so $T_0=0$. The Strong Markov property at successive first passages to lower levels and part (a) giveAt $D_n=1$, the next state is zero with probability $r/2$, one with probability $1/2$, and two with probability $(1-r)/2$. First-step analysis therefore givesorSolving,Substituting $\mu =2/(2q-1)$ now yieldswhich is the required expected capture time.