 Solution

Starting from a state $i$, let ${\tau }_i=\inf \{n\ge 1:X_n=i\}$ be the first return time, with value infinity if no return occurs. The state is recurrent if ${\mathrm{Pr}}_i({\tau }_i<\infty )=1$ and transient if this probability is less than 1. A recurrent state is positive recurrent if ${\mathbb{E}}_i{\tau }_i<\infty$, and null recurrent if that expectation is infinite.

If $i$ and $j$ communicate, choose $r,s$ with $p_{ij}^{(r)}>0$ and $p_{ji}^{(s)}>0$. The Chapman-Kolmogorov equations give, for every $n$,If $j$ is recurrent, the supplied criterion makes ${\sum }_n{p}_{jj}^{(n)}$ infinite. Summing this inequality shows that ${\sum }_n{p}_{ii}^{(n)}$ is also infinite, so $i$ is recurrent. The same argument with the states exchanged gives the converse. Hence recurrence and transience are class properties: states in one communicating class cannot have different answers to that dichotomy.