Past exam of the mathematics course of the University of Cambridge / 2006 / ib / Paper 1 / 19C / Solution

For the natural filtration ${\mathcal{F}}_n=\sigma (X_0,\dots ,X_n)$, a stopping time $T$ takes values in $\{0,1,\dots \}\cup \{\infty \}$ and satisfies $\{T\le n\}\in {\mathcal{F}}_n$ for each $n$. Whether it has occurred by time $n$ can be decided from the observations through that time.

The Strong Markov property for a time-homogeneous Markov chain says that, on $\{T<\infty \}$, conditional on ${\mathcal{F}}_T$, the process $(X_{T+r}{)}_{r\ge 0}$ has the law of a fresh chain started at $X_T$. In particular its future conditional law depends on the past only through $X_T$. For example $\mathbb{P}(X_{T+r}=j\mid {\mathcal{F}}_T)=(P^r{)}_{X_{T}j}$ on that event, and the corresponding statement holds for every finite future path.

Start at $i$ and let $T_0=0$, $T_{r+1}=\inf \{n>T_r:X_n=i\}$, with $\inf \varnothing =\infty$. These are stopping times. Put $f={\mathbb{P}}_i(T_1<\infty )$. Each finite return restarts the chain at $i$, so the Strong Markov property gives inductivelyThe events decrease as $r$ increases; infinitely many visits occur exactly when every return time is finite. Continuity of probability therefore givesNo irreducibility assumption is needed for this zero-one conclusion at the starting state.

Let $N={\sum }_{n\ge 0}{1}_{\{X_n=i\}}$, counting the initial visit. Summing nonnegative random variables and then using the tail-sum identity givesIf $f<1$, this is $1/(1-f)<\infty$. Thus divergence of the given sum forces $f=1$, and infinitely many visits then occur with probability one. This proves the recurrence criterion by return probabilities directly from the regeneration at return times; divergent probability sums alone would not suffice for arbitrary dependent events.