Past exam of the mathematics course of the University of Cambridge / 2016 / ib / Paper 4 / 9H (Markov Chains)

If there are $i$ balls in box A, a ball currently in B is selected with probability $(k-i)/k$, increasing the count, and a ball in A is selected with probability $i/k$, decreasing it. For $k\ge 1$,with every other transition zero. This is the Ehrenfest urn model.

The stationary distribution isIt is normalized by the binomial theorem, and it obeys the detailed balance:All other pairs have zero flux in both directions. These identities prove stationarity and show that this is a reversible Markov chain in equilibrium. The finite chain is irreducible and has period two; reversibility does not require aperiodicity.

A finite irreducible Markov chain is positive recurrent. Its mean recurrence time to state $i$, starting at $i$ and counting the first positive return, is $1/{\pi }_i$. This identity comes from the long-run fraction of time spent at that state: successive return cycles each contain one visit at their start, so the fraction is the reciprocal of the mean cycle length. It remains valid for a periodic chain.

Here ${\pi }_0=2^{-k}$, henceFor $k=1$ the return is deterministically after two minutes, which checks the formula.