Past exam of the probability, statistics and options I course of the University of Oxford / 2008 / M / 2 / Solution

The Markov property says that, conditional on the present state, the next state is independent of the earlier history:Two states communicate if each is reachable from the other with positive probability in finitely many steps. The equivalence classes are communicating classes; a class is closed if no positive-probability transition leaves it.

The displayed probabilities require $0\le p\le 1/2$; negative values allowed by the printed one-sided inequality are not transition probabilities. For $p=0$, every state is an absorbing singleton, so all singleton classes are closed. For $0<p\le 1/2$, neighboring states communicate in both directions, so there is one closed irreducible class $S$ and no open classes.

For $m=3$, the detailed balance relations give $2{\pi }_0={\pi }_1={\pi }_2=2{\pi }_3$. Normalization yieldsFor $0<p<1/2$, positive holding probabilities make the finite irreducible chain aperiodic, so its transition probabilities converge to ${\pi }_j$ from every initial state. At $p=1/2$, every step changes parity and the period is two; convergence from a fixed initial state fails. A stationary initial mixture still has distribution $\pi$ at every time, but this does not give convergence of the transition probabilities asked for.