= Solution
For the <countable-state Markov chain ergodic theorem>, let the <Markov chain> be irreducible and positive recurrent, with its unique <stationary distribution> $\pi$. From any initial state, for every real $f$ satisfying $\sum_j\pi_j|f(j)|<\infty$,
$$
\boxed{\frac1n\sum_{m=0}^{n-1}f(X_m)\longrightarrow\sum_j\pi_jf(j)\quad\text{almost surely}.}
$$
In particular, $n^{-1}\sum_{m=0}^{n-1}\mathbf1_{\{X_m=j\}}\to\pi_j$ <almost surely>. This time-average conclusion requires no <aperiodicity>.
The <countable-state Markov chain convergence theorem> adds the hypothesis of an <aperiodic Markov chain> and concludes
$$
\boxed{P^n(i,j)\longrightarrow\pi_j\quad\text{for every }i,j.}
$$
It also follows that $\mathbb P_\nu(X_n=j)\to\pi_j$ for every initial <probability distribution> $\nu$. For an irreducible recurrent chain, the occupation-frequency theorem can also be stated as a limit $1/\mathbb E_jT_j^+$, interpreted as zero for a <null recurrent state>; in the positive recurrent case this equals $\pi_j$.
Back to article page
