Past exam of the options II course of the University of Oxford / 2006 / N / 2 / i

The state space is $\{0,1,2,\dots \}$. Write $q=1-p$. The nonzero transition probabilities areThe stationary equations give ${\pi }_1={\pi }_0$ and ${\pi }_j=q^{j-1}{\pi }_0$ for $j\ge 1$. Normalization gives ${\pi }_0[1+{\sum }_{j\ge 1}{q}^{j-1}]={\pi }_0(1+1/p)=1$, soThe remaining equation ${\pi }_0=p{\sum }_{j\ge 1}{\pi }_j$ is satisfied, confirming stationarity.

The chain is irreducible: every positive state can jump to zero, and zero can reach every state through consecutive immigration steps. A stationary probability distribution makes it positive recurrent. Returns to zero can have lengths two and three with positive probability, so it is also aperiodic. The Markov-chain ergodic theorem now gives the almost-sure long-run time proportion for $X_n>K$ as the stationary tail:This time-average conclusion applies to the specified start at zero.