Past exam of the probability, statistics and options I course of the University of Oxford / 2009 / M / 2 / c

After the first step, a chain starting at one enters the closed class $\{2,3\}$ with probability $p$, starting at two, or enters $\{4,5\}$ with probability $1-p$ and can never reach three. Within the first class, the probability of going from its first state to its second in $n-1$ steps is the complement of the formula in part (b). ThusAt $n=1$ this correctly gives zero.

The total eventual mass of the closed class $\{4,5\}$ is ${\lambda }_4+{\lambda }_5+(1-p){\lambda }_1$. The other closed class cannot contribute to state five. Since $|1-a-b|<1$, the two-state chain within the last class converges to its stationary distribution, with state-five mass $a/(a+b)$. Therefore