A <recurrent state> $i$ satisfies $\Pr_i(T_i^+<\infty)=1$, where $T_i^+=\inf\{n\geq1:X_n=i\}$. Saying that the <transition matrix> is recurrent means that every state is recurrent. In an <irreducible Markov chain> it suffices that one state is recurrent, since recurrence is a communicating-class property.

Solution

Past exam of the mathematics course of the University of Cambridge / 2004 / ib / Paper 1 / 11H / b / Solution

 Solution (source code)