Past exam of the options I course of the University of Oxford / 2005 / N / 1 / Solution

States $i,j$ communicate when each is accessible from the other: $(P^m{)}_{ij}>0$ and $(P^n{)}_{ji}>0$ for some nonnegative integers $m,n$. The communicating classes areIndeed $1$ and $4$ move only between themselves; $2\to 5\to 3\to 2$ links the second class, but it has exits to $1,4,6$ and no return from the closed classes; state 6 is absorbing.

Let $h_i$ be the probability of ever reaching 6 from state $i$. Then $h_1=h_4=0$, $h_6=1$, and first-step conditioning givesThus $h_3=h_2=3h_5/4$ and $3h_5=h_3+1$, giving $h_5=4/9$. Therefore