Past exam of the mathematics course of the University of Cambridge / 2021 / ib / Paper 1 / 19H (Markov Chains)

A random time $T$ is a stopping time when the event $\{T\le n\}$ is determined by $X_0,\dots ,X_n$. The Strong Markov property says that, conditionally on $T<\infty$ and $X_T=x$, the process $(X_{T+k}{)}_{k\ge 0}$ is a fresh Markov chain started at $x$, independent of the history before $T$.

Use the state space $\{1,2,4\}\times {\mathbb{Z}}_{\ge 0}$, with $(2,0)$ absorbing. Observe the chain only when it is at square 2. From $(2,k)$, the change $Y$ in wealth by the next return to square 2 has distributionIndeed, heads lands on square 3 and returns to square 2 after losing £1. After tails reaches square 4, the remaining two or three moves give the other cases.

For $r=2/3$,Thus $r^{M_j}$ is a martingale for the embedded wealth random walk $M_j$. Stopping when it first reaches $m-1$ or a large upper level and then letting that level tend to infinity givesThe upper-bound contribution vanishes because $0<r<1$; equivalently, this is the smaller probability solution of the first-step equation.

By part (a) and repeated use of the Strong Markov property, the probability of descending from $(2,k)$ to $(2,0)$ isStarting from square 1 with £$m$, heads moves to $(2,m)$, whereas tails lands on square 3, loses £1, and moves to $(2,m-1)$. Therefore the loss probability is