 Solution

For the countable-state Markov chain ergodic theorem, let the Markov chain be irreducible and positive recurrent, with its unique stationary distribution $\pi$. From any initial state, for every real $f$ satisfying ${\sum }_j{\pi }_j|f(j)|<\infty$,In particular, $n^{-1}{\sum }_{m=0}^{n-1}{1}_{\{X_m=j\}}\to {\pi }_j$ almost surely. This time-average conclusion requires no aperiodicity.

The countable-state Markov chain convergence theorem adds the hypothesis of an aperiodic Markov chain and concludesIt also follows that ${\mathbb{P}}_{\nu }(X_n=j)\to {\pi }_j$ for every initial probability distribution $\nu$. For an irreducible recurrent chain, the occupation-frequency theorem can also be stated as a limit $1/{\mathbb{E}}_j{T}_j^{+}$, interpreted as zero for a null recurrent state; in the positive recurrent case this equals ${\pi }_j$.