Past exam of the options II course of the University of Oxford / 2011 / M / M5 / b / ii / Solution

Let $Q=H_N$ and defineThe probabilities sum to $Q^{-1}(1+{\sum }_{k=1}^{N-1}(k+1{)}^{-1})=1$. For $k<N$, the incoming probability isusing a telescoping sum. Similarly,Thus $\pi$ is a stationary distribution. Every state can reach $N$ in one step, and $N$ can reach every lower positive state in one step, so the chain is irreducible. The self-loop at $N$ has probability $1/N>0$, hence the chain is aperiodic. The convergence theorem for a finite irreducible aperiodic Markov chain now givesFor $N=1$, the conclusion follows directly from the single absorbing/refilled state.

This is the limiting law of the uniform depletion and refill Markov chain.