= Solution
Start the <Markov chain> at a state $i$ and define the strictly positive <first return time> $T_i^+=\inf\{n\geq1:X_n=i\}$. A <recurrent state> satisfies $\mathbb P_i(T_i^+<\infty)=1$; a <transient state> has this <probability> strictly below one. Equivalently a <recurrent state> is visited infinitely often <almost surely> when started there, whereas a <transient state> receives finitely many visits <almost surely>. The <recurrence criterion by return probabilities> gives the equivalent test
$$
i\text{ recurrent}\iff\sum_{n=0}^{\infty}p_{ii}^{(n)}=\infty.
$$
States in a <communicating class> share this classification. In an <irreducible Markov chain> all states communicate, so the chain is called a <recurrent Markov chain> if all its states are recurrent and a <transient Markov chain> if all are transient.
Back to article page
