= Reset-or-increment Markov chain
{title2=$P(0,1)=1,\quad P(k,k+1)=p,\quad P(k,0)=1-p\quad(k\geq1)$}
For $0<p<1$, consider the <Markov chain> on the nonnegative integers with the displayed <transition probabilities>. From zero it starts a new upward run; from each positive state it either increments or resets. Writing $q=1-p$, its <stationary distribution> is
$$
\pi_0=\frac{q}{1+q},\qquad \pi_k=\frac{q}{1+q}p^{k-1}\quad(k\geq1).
$$
The balance equations give $\pi_1=\pi_0$ and $\pi_k=p\pi_{k-1}$ for $k\geq2$; the <geometric series> normalizes the result, and the zero-state equation $\pi_0=q(1-\pi_0)$ agrees. It is an <irreducible Markov chain> and is <positive recurrent>. Equivalently, its <first return time> to zero is one plus a <geometric random variable> with success probability $q$, so its <mean recurrence time> is $1+1/q$. Return paths of lengths two and three also make it <aperiodic>.
Back to article page
