Past exam of the mathematics course of the University of Cambridge / 2025 / ib / Paper 4 / 7H (Markov Chains)

The communicating classes are $\{1,3\}$ and $\{2,4\}$. The first is open because state 3 can enter 4 and cannot return; the second is closed.

On the closed class, the transition matrix isIts stationary distribution is $(3/11,8/11)$. The class is irreducible and aperiodic because of its self-loops, while the other class is transient and enters it almost surely. Hence