Past exam of the mathematics course of the University of Cambridge / 2022 / ib / Paper 2 / 18H (Markov Chains)

The chain is an irreducible birth-death process. The detailed balance equations areso ${\pi }_{i+1}={\pi }_i/2$. Normalization gives the stationary distributionAn irreducible countable-state Markov chain that possesses a stationary probability distribution is positive recurrent. Hence $X$ is positive recurrent.

The self-loop at zero makes the irreducible chain aperiodic. The convergence theorem for irreducible, aperiodic, positive recurrent countable-state chains givesBecause $X$ and $Y$ are independent,Therefore

The pair $Z_n=(X_n,Y_n)$ is an irreducible positive recurrent product Markov chain with stationary distributionThe stationary cycle occupation formula says that, during one return cycle to a state $z$, the expected number of visits to a set $A$ is $\Pi (A)/\Pi (z)$. TakeThenThe initial and terminal states both have $Y=0$, so either convention for including the endpoints gives the same count. Thus