= Solution
{c}
The <state space> is $\{0,1,2,\ldots\}$. Write $q=1-p$. The nonzero transition probabilities are
$$
P(0,1)=1,\qquad P(j,0)=p,\quad P(j,j+1)=q\quad(j\geq1).
$$
The stationary equations give $\pi_1=\pi_0$ and $\pi_j=q^{j-1}\pi_0$ for $j\geq1$. Normalization gives $\pi_0[1+\sum_{j\geq1}q^{j-1}]=\pi_0(1+1/p)=1$, so
$$
\boxed{\pi_0=\frac p{1+p},\qquad
\pi_j=\frac{p(1-p)^{j-1}}{1+p}\quad(j\geq1).}
$$
The remaining equation $\pi_0=p\sum_{j\geq1}\pi_j$ is satisfied, confirming stationarity.
The chain is <irreducible>: every positive state can jump to zero, and zero can reach every state through consecutive immigration steps. A stationary probability distribution makes it <positive recurrent>. Returns to zero can have lengths two and three with positive probability, so it is also <aperiodic>. The Markov-chain ergodic theorem now gives the almost-sure long-run time proportion for $X_n>K$ as the stationary tail:
$$
\boxed{\sum_{j=K+1}^\infty\pi_j=\frac{(1-p)^K}{1+p}.}
$$
This time-average conclusion applies to the specified start at zero.
Back to article page
