= Countable-state Markov chain ergodic theorem
{title2=$n^{-1}\sum_{m=0}^{n-1}f(X_m)\to\sum_i\pi_i f(i)$}
For an <irreducible Markov chain> on a countable <state space> with <positive recurrence> and <stationary distribution> $\pi$, the displayed <almost sure convergence> holds from every initial state whenever $\sum_i\pi_i|f(i)|<\infty$. In particular, the empirical fraction of visits to $i$ converges to $\pi_i$. No <aperiodicity> is needed. Regeneration at successive returns to a state reduces the result to the <strong law of large numbers> for independent excursion lengths and rewards. This extends the finite-state <Markov chain ergodic theorem>.
Back to article page
