Past exam of the mathematics course of the University of Cambridge / 2026 / ib / Paper 1 / 18H (Markov Chains)

A stopping time $T$ has $\{T\le n\}$ determined by $X_0,\dots ,X_n$. The Strong Markov property says that, conditional on $T<\infty$ and $X_T=i$, the post-$T$ chain is a fresh chain started at $i$, independent of the pre-$T$ history.

First-step equations $h_B=1+h_W/2$ and $h_W=1+2h_B/3$ give $h_B=9/4$.

Augment by whether the last sound was a growl. The equations $h_B=1+(h_G+h_W)/2$, $h_G=1+2h_B/3$, $h_W=1+2h_B/3+h_G/3$ give $h_B=39/4$.

Augment the state by the current sound and $D=N_b-N_w$, and solve the translation-invariant first-step equations. Equivalently, solve the Poisson equation for the additive reward $+1,0,-1$. The stationary probabilities are $(2/5,3/10,3/10)$, so the drift is $1/10$ per sound. The correction term has the same value at the initial and terminal bark states; optional stopping at $D=m$ therefore gives the expected produced length $10m$.