Past exam of the mathematics course of the University of Cambridge / 2026 / ib / Paper 1 / 18H / c / Solution

Augment the state by the current sound and $D=N_b-N_w$, and solve the translation-invariant first-step equations. Equivalently, solve the Poisson equation for the additive reward $+1,0,-1$. The stationary probabilities are $(2/5,3/10,3/10)$, so the drift is $1/10$ per sound. The correction term has the same value at the initial and terminal bark states; optional stopping at $D=m$ therefore gives the expected produced length $10m$.