Past exam of the probability, statistics and options I course of the University of Oxford / 2009 / M / 2 / c / i / Solution

After the first step, a chain starting at one enters the closed class $\{2,3\}$ with probability $p$, starting at two, or enters $\{4,5\}$ with probability $1-p$ and can never reach three. Within the first class, the probability of going from its first state to its second in $n-1$ steps is the complement of the formula in part (b). ThusAt $n=1$ this correctly gives zero.