= Solution
{c}
After the first step, a chain starting at one enters the closed class $\{2,3\}$ with probability $p$, starting at two, or enters $\{4,5\}$ with probability $1-p$ and can never reach three. Within the first class, the probability of going from its first state to its second in $n-1$ steps is the complement of the formula in part (b). Thus
$$
\boxed{\mathbb P(X_n=3\mid X_0=1)
=p\frac a{a+b}\left[1-(1-a-b)^{n-1}\right],\qquad n\geq1.}
$$
At $n=1$ this correctly gives zero.
Back to article page
