Past exam of the mathematics course of the University of Cambridge 2024 ib Paper 1 19H a Solution Created 2026-09-24 Updated 2026-09-25
Starting from , the probability of reaching before returning to is . Conditional on reaching , each visit to is followed by a hit on before the next return to with probability , by symmetry and the Strong Markov property. For , the number of visits after entry is therefore geometric on with mean . Hence the two-state excursion visit law givesIn the degenerate case , the chain never reaches before its return to , so almost surely and .
Stationary cycle occupation formula Created 2026-09-24 Updated 2026-11-05
For an irreducible positive recurrent Discrete-time Markov chain with Stationary distribution , fix a state and its positive first return . The Stationary cycle occupation formula isThe cycle includes the initial visit to and excludes the returning visit. Thus for the left side is exactly one, consistent with the ratio.
Successive return cycles are independent with the same law by the Strong Markov property. The renewal reward ratio gives the long-run visit fraction as expected cycle occupation divided by expected cycle length. Kac's lemma supplies , so the ergodic visit fraction gives the displayed identity. positive recurrence ensures the cycle length and all its occupation rewards are integrable.
For distinct , the Two-state excursion visit law gives the same expected occupation as , where and . ConsequentlyThis identity does not require reversibility: it follows from return-cycle occupation. It also does not require aperiodicity, since time averages still converge in a periodic positive recurrent chain.