= Markov chain ergodic theorem
{c}
For an irreducible finite <Markov chain> with <stationary distribution> $\pi$, the fraction of visits to any state $j$ converges almost surely to $\pi_j$, for every initial state. More generally, the empirical average of a function of the state converges to its average under $\pi$. Aperiodicity is needed for convergence of the one-time distributions, but not for this empirical-frequency conclusion. Regeneration at returns to a state proves the conclusion by applying the strong law to independent excursion lengths and excursion rewards.
Back to article page
