Past exam of the options I course of the University of Oxford / 2006 / N / N1

Two states lie in the same communicating class when each is reachable from the other by transitions of positive probability. Inspecting the directed transitions gives three classes:State 1 has no incoming transition from another state and can leave, so its singleton class is open and transient. More explicitly, the probability of returning to 1 after time zero is only $0.1$, since a departure can never be reversed.

States $2,3,6,7$ communicate: $2\to 3\to 7\to 2$ and $2\to 6\to 7$ are positive-probability paths. The class is open because, for example, $2\to 8$ and $6\to 4$ leave it; no transition from the class $\{4,5,8\}$ returns to it. Every state of $\{2,3,6,7\}$ is transient. One can see eventual exit directly: from each of its four states there is a positive-probability path of at most three steps to 8; the minimum of these four path probabilities is positive, so survival through repeated three-step blocks has a geometrically decreasing upper bound.

Finally $4\to 8\to 5\to 4$ shows that $\{4,5,8\}$ communicates, and every outgoing transition stays inside it. It is a closed finite irreducible class, so all its states are positive recurrent. Therefore the first two classes are open and transient; the last is closed and recurrent.

A stationary distribution is a nonnegative row vector $\pi$ with $\pi P=\pi$ and ${\sum }_i{\pi }_i=1$. The balance equations giveThe first yields ${\pi }_1={\pi }_4$. Substitution into the middle pair yields ${\pi }_2=3{\pi }_4/4$ and ${\pi }_3={\pi }_4$; the fourth equation is then satisfied. Normalization gives $15{\pi }_4/4=1$, henceAll states communicate, so this is the unique stationary distribution. The self-loop at state 1 additionally makes this finite Markov chain aperiodic, although aperiodicity is not required for the balance calculation.