= Uniform depletion and refill Markov chain
Refill the stock to a fixed positive size $N$ immediately whenever a step empties it. The resulting <Markov chain> has <state space> $\{1,\ldots,N\}$ and transitions $P(i,j)=1/i$ for $j<i$ and $P(i,N)=1/i$. It is <irreducible Markov chain>[irreducible] and <aperiodic Markov chain>[aperiodic], since every state reaches $N$, $N$ reaches every state, and $P(N,N)>0$. Its unique <stationary distribution> is $\pi_N=1/H_N$ and $\pi_k=1/((k+1)H_N)$ for $k<N$. Normalization is the <harmonic number> identity. For $k<N$, stationarity follows from $\sum_{j=k+1}^{N-1}1/(j(j+1))+1/N=1/(k+1)$; the same telescoping sum starting at $1$ gives stationarity at $N$. The finite-state convergence theorem therefore makes this <stationary distribution> the limiting distribution from every initial state.
Back to article page
