For an irreducible positive recurrent Discrete-time Markov chain with Stationary distribution , fix a state and its positive first return . The Stationary cycle occupation formula is
The cycle includes the initial visit to and excludes the returning visit. Thus for the left side is exactly one, consistent with the ratio.
Successive return cycles are independent with the same law by the Strong Markov property. The renewal reward ratio gives the long-run visit fraction as expected cycle occupation divided by expected cycle length. Kac's lemma supplies , so the ergodic visit fraction gives the displayed identity. positive recurrence ensures the cycle length and all its occupation rewards are integrable.
For distinct , the Two-state excursion visit law gives the same expected occupation as , where and . Consequently
This identity does not require reversibility: it follows from return-cycle occupation. It also does not require aperiodicity, since time averages still converge in a periodic positive recurrent chain.

Articles by others on the same topic (0)

There are currently no matching articles.