A Recurrent Markov chain returns to its states rather than eventually leaving them forever. A state is recurrent if its positive first return time satisfies . A chain is recurrent when every state is recurrent. Within an irreducible class, recurrence is shared by every state, so testing one state suffices.
The Strong Markov property at successive return times shows that a Recurrent state is visited infinitely often almost surely when starting there. If its return Probability is , the number of visits including time zero has a geometric distribution on the positive integers, with mean . Consequently the Green-function criterion isTonelli's theorem identifies this sum with the expected number of visits. recurrence does not require a finite mean return time: null recurrence and positive recurrence distinguish infinite from finite mean recurrence time. simple symmetric random walk on the integers is null recurrent, whereas every finite irreducible chain is positive recurrent.
An irreducible Markov chain is null recurrent when every state is a null recurrent state: it returns to that state almost surely, but its mean recurrence time is infinite. Such a countable-state chain has no stationary distribution.
An irreducible Markov chain is positive recurrent when its mean return times are finite. For a nonexplosive Markov chain on a countable state space, this is equivalent to the existence of a stationary distribution on its irreducible class. For continuous time, a return cycle includes departure and return, rather than counting immediate residence as a return.
For an irreducible Markov chain on a countable state space with positive recurrence and stationary distribution , the displayed almost sure convergence holds from every initial state whenever . In particular, the empirical fraction of visits to converges to . No aperiodicity is needed. Regeneration at successive returns to a state reduces the result to the strong law of large numbers for independent excursion lengths and rewards. This extends the finite-state Markov chain ergodic theorem.
In the standard discrete-time state terminology, an ergodic state is a positive recurrent state with aperiodicity. Return occurs almost surely, the mean strictly positive return time is finite, and the greatest common divisor of possible return lengths is one. An irreducible chain shares these properties among all its states.
Articles by others on the same topic
There are currently no matching articles.