Past exam of the probability course of the University of Oxford / 2017 / 3 / a

A transition matrix $P$ defines an irreducible Markov chain if every state is accessible from every other state:Equivalently, all states belong to one communicating class. When $i\ne j$, a positive number of steps is necessarily used.

A stationary distribution is a nonnegative row vector $\pi =({\pi }_i{)}_{i\in S}$ with total mass one that is preserved by one transition:In matrix notation, $\pi P=\pi$. If the initial state has distribution $\pi$, so does the state at each later time.

For an irreducible Markov chain on a countable state space, the set has either zero or one element. By uniqueness of stationary distributions for an irreducible countable chain, a stationary probability distribution exists exactly in the positive recurrent case, and thenfor the first return time $T_i^{+}$. The right-hand side fixes every entry, proving uniqueness.

For a finite state space, existence is guaranteed, so there is exactly one. A countably infinite state space can have none: the simple symmetric random walk on the integers is irreducible and null recurrent. Periodicity does not affect this existence-and-uniqueness statement.