Recurrent Markov chain

A state is recurrent when the chain returns to it almost surely after starting there.

An irreducible Markov chain is positive recurrent when its mean return times are finite. For a nonexplosive Markov chain on a countable state space, this is equivalent to the existence of a stationary distribution on its irreducible class. For continuous time, a return cycle includes departure and return, rather than counting immediate residence as a return.

A recurrent state is positive recurrent when its expected return time is finite.

In the standard discrete-time state terminology, an ergodic state is a positive recurrent state with aperiodicity. Return occurs almost surely, the mean strictly positive return time is finite, and the greatest common divisor of possible return lengths is one. An irreducible chain shares these properties among all its states.

A recurrent state is null recurrent when its expected return time is infinite.

A state is recurrent exactly when the sum over time of its return probabilities diverges.

For nearest-neighbour random walk on ${\mathbb{Z}}_{+}$ with upward probability $p>q$, the probability of ever hitting zero from $i$ is $(q/p{)}^i$.