Past exam of the Probability, statistics and options II course of the University of Oxford / 2011 / M / M5 / b

Since refilling is immediate, the observed arrival-state space becomes $\{1,\dots ,N\}$. A zero remainder is replaced by $N$, while all positive remainders are unchanged. ThusIn particular state one moves to $N$ with probability one, and state $N$ has a self-loop of probability $1/N$.

Write $Q={\sum }_{m=1}^{N}1/m$ and propose ${\pi }_N=1/Q$, ${\pi }_j=1/((j+1)Q)$ for $j<N$. These probabilities sum to one because $1+{\sum }_{j=1}^{N-1}1/(j+1)=Q$.

For $j<N$, the incoming stationary probability isusing the telescoping identity $1/(k(k+1))=1/k-1/(k+1)$. Similarly ${\sum }_{k=1}^N{\pi }_k/k=1/Q={\pi }_N$. Thus $\pi$ is stationary.

The finite chain is irreducible: every state can reach $N$, and from $N$ every state is reachable in one step. Its self-loop at $N$ makes it aperiodic. The finite-state convergence theorem therefore gives the claimed limit from any initial state:For $N=1$ this is the single absorbing refill state and the same formula gives probability one.