Past exam of the options I course of the University of Oxford / 2009 / M / M2 / c

At the first step, the chain enters state two with probability $p$ and state four with probability $1-p$. The classes $\{2,3\}$ and $\{4,5\}$ are closed and disjoint, so only the first branch can reach state three. On $\{2,3\}$ the transition matrix is exactly the two-state matrix from part (b). Thus after the remaining $n-1$ steps, the probability of the second state in that class isFor $n=1$ it is zero, as the original matrix requires; this statement covers the zero-base exponent case without ambiguity.

State one is left after one step and never revisited. The eventual probability mass in the closed class $\{4,5\}$ isWithin this class the stationary distribution is $(b/(a+b),a/(a+b))$, because the balance equation is $a{\pi }_4=b{\pi }_5$. The two-state recurrence shows convergence from either starting state, since $|1-a-b|<1$. The other closed class never reaches state five. Therefore