Past exam of the options I course of the University of Oxford / 2006 / N / N1 / i / Solution

Two states lie in the same communicating class when each is reachable from the other by transitions of positive probability. Inspecting the directed transitions gives three classes:State 1 has no incoming transition from another state and can leave, so its singleton class is open and transient. More explicitly, the probability of returning to 1 after time zero is only $0.1$, since a departure can never be reversed.

States $2,3,6,7$ communicate: $2\to 3\to 7\to 2$ and $2\to 6\to 7$ are positive-probability paths. The class is open because, for example, $2\to 8$ and $6\to 4$ leave it; no transition from the class $\{4,5,8\}$ returns to it. Every state of $\{2,3,6,7\}$ is transient. One can see eventual exit directly: from each of its four states there is a positive-probability path of at most three steps to 8; the minimum of these four path probabilities is positive, so survival through repeated three-step blocks has a geometrically decreasing upper bound.

Finally $4\to 8\to 5\to 4$ shows that $\{4,5,8\}$ communicates, and every outgoing transition stays inside it. It is a closed finite irreducible class, so all its states are positive recurrent. Therefore the first two classes are open and transient; the last is closed and recurrent.