Past exam of the options I course of the University of Oxford / 2009 / M / M2 / c / i / Solution

At the first step, the chain enters state two with probability $p$ and state four with probability $1-p$. The classes $\{2,3\}$ and $\{4,5\}$ are closed and disjoint, so only the first branch can reach state three. On $\{2,3\}$ the transition matrix is exactly the two-state matrix from part (b). Thus after the remaining $n-1$ steps, the probability of the second state in that class isFor $n=1$ it is zero, as the original matrix requires; this statement covers the zero-base exponent case without ambiguity.