An irreducible nonexplosive continuous-time Markov chain has an equilibrium probability distribution exactly when it is positive recurrent. The distribution is unique, and gives the long-run fraction of elapsed time in each state. A null recurrent chain has no equilibrium probability distribution. The finite one-state absorbing case is trivially stationary; the return-cycle descriptions concern nontrivial irreducible classes.
A state of an irreducible continuous-time Markov chain is transient if, after departure, its return probability is less than one. Equivalently its jump chain visits that state only finitely often almost surely. This is a class property.
A state is recurrent if, after departure, its return probability is one. For an irreducible nonexplosive continuous-time Markov chain, this is equivalent to recurrence of its jump chain and is shared by all states.
A recurrent state is positive recurrent if its expected elapsed return-cycle time is finite. The cycle starts on entering the state and ends at its next entrance after a departure; it includes the initial holding time. This convention avoids counting the immediate return caused by a right-continuous path remaining at its initial state.
A recurrent state is null recurrent if its expected elapsed return-cycle time is infinite. Recurrence itself agrees with that of the jump chain, but positive recurrence need not agree, because large states can have very short holding times.
Articles by others on the same topic
There are currently no matching articles.