= Solution
For the natural <filtration> $\mathcal F_n=\sigma(X_0,\ldots,X_n)$, a <stopping time> $T$ takes values in $\{0,1,\ldots\}\cup\{\infty\}$ and satisfies $\{T\le n\}\in\mathcal F_n$ for each $n$. Whether it has occurred by time $n$ can be decided from the observations through that time.
The <Strong Markov property> for a time-homogeneous <Markov chain> says that, on $\{T<\infty\}$, conditional on $\mathcal F_T$, the process $(X_{T+r})_{r\ge0}$ has the law of a fresh chain started at $X_T$. In particular its future conditional law depends on the past only through $X_T$. For example $\mathbb P(X_{T+r}=j\mid\mathcal F_T)=(P^r)_{X_Tj}$ on that event, and the corresponding statement holds for every finite future path.
Start at $i$ and let $T_0=0$, $T_{r+1}=\inf\{n>T_r:X_n=i\}$, with $\inf\varnothing=\infty$. These are <stopping times>. Put $f=\mathbb P_i(T_1<\infty)$. Each finite return restarts the chain at $i$, so the <Strong Markov property> gives inductively
$$
\mathbb P_i(T_r<\infty)=f^r.
$$
The events decrease as $r$ increases; infinitely many visits occur exactly when every return time is finite. Continuity of probability therefore gives
$$
\boxed{\mathbb P_i(\text{infinitely many visits to }i)=\lim_{r\to\infty}f^r
=\begin{cases}0,&f<1,\\1,&f=1.\end{cases}}
$$
No irreducibility assumption is needed for this zero-one conclusion at the starting state.
Let $N=\sum_{n\ge0}\mathbf1_{\{X_n=i\}}$, counting the initial visit. Summing nonnegative random variables and then using the tail-sum identity gives
$$
\sum_{n\ge0}\mathbb P_i(X_n=i)=\mathbb E_iN
=\sum_{r\ge0}\mathbb P_i(N\ge r+1)=\sum_{r\ge0}f^r.
$$
If $f<1$, this is $1/(1-f)<\infty$. Thus <divergence> of the given sum forces $f=1$, and \b[infinitely many visits then occur with probability one]. This proves the <recurrence criterion by return probabilities> directly from the regeneration at return times; divergent probability sums alone would not suffice for arbitrary dependent events.
Back to article page
