 Solution

A Markov chain is irreducible if every state communicates with every other: for any $i,j$ there is an integer $n\ge 0$ with $p_{ij}^{(n)}>0$. Thus any state can be reached from any other in finitely many steps with positive probability.