= Solution
At the first step, the chain enters state two with probability $p$ and state four with probability $1-p$. The classes $\{2,3\}$ and $\{4,5\}$ are closed and disjoint, so only the first branch can reach state three. On $\{2,3\}$ the <transition matrix> is exactly the two-state <matrix> from part (b). Thus after the remaining $n-1$ steps, the probability of the second state in that class is
$$
\boxed{\mathbb P(X_n=3\mid X_0=1)
=\frac{pa}{a+b}\left(1-(1-a-b)^{n-1}\right)},\qquad n\geq1.
$$
For $n=1$ it is zero, as the original <matrix> requires; this statement covers the zero-base exponent case without ambiguity.
Back to article page
