Past exam of the options II course of the University of Oxford / 2011 / M / M5 / b

Use the state space $\{1,\dots ,N\}$, since an empty box is refilled immediately. The probability that formerly led from $i$ to $0$ now leads to $N$. Thus the transition matrix isFor $i=N$, the second line includes the self-loop obtained when all stickers are taken. When $N=1$, this is simply the one-state Markov chain.

Let $Q=H_N$ and defineThe probabilities sum to $Q^{-1}(1+{\sum }_{k=1}^{N-1}(k+1{)}^{-1})=1$. For $k<N$, the incoming probability isusing a telescoping sum. Similarly,Thus $\pi$ is a stationary distribution. Every state can reach $N$ in one step, and $N$ can reach every lower positive state in one step, so the chain is irreducible. The self-loop at $N$ has probability $1/N>0$, hence the chain is aperiodic. The convergence theorem for a finite irreducible aperiodic Markov chain now givesFor $N=1$, the conclusion follows directly from the single absorbing/refilled state.

This is the limiting law of the uniform depletion and refill Markov chain.