Past exam of the options II course of the University of Oxford / 2004 / N / N2 / b / ii / Solution

For an irreducible positive recurrent Markov chain, the Markov chain ergodic theorem gives a unique stationary distribution $\pi$, with almost-sure long-run occupation fraction ${\pi }_j$ in state $j$. More generally, time averages of an integrable state reward converge to its stationary mean. If the chain is also aperiodic, then ${\mathbb{P}}_i(X_n=j)\to {\pi }_j$ for every initial state $i$; the aperiodicity assumption is needed for this last distributional conclusion.

For either transition rule in part (i), $0<p<1$ permits progression through the phase-A states and a return from phase B to zero. Both chains are finite and irreducible, hence positive recurrent. State $K$ has a positive self-loop, so both are aperiodic. Thereforeusing respectively the cumulative or consecutive stationary distribution given in part (i), for every starting distribution.

The excluded endpoint cases are simple: at $p=0$, phase B is eventually reached and then absorbing, so the limiting law is the point mass at $K$. At $p=1$, the cumulative rule traps the chain at its current phase-A count (or eventually at zero if started in phase B), so the limit can depend on the initial count; under the consecutive rule zero is eventually absorbing. Irreducibility and the preceding stationary formulas must not be asserted unchanged at these endpoints.