Past exam of the mathematics course of the University of Cambridge / 2025 / ib / Paper 3 / 8H (Markov Chains)

A state is recurrent when, starting there, the chain returns to it with probability one. An irreducible chain is recurrent when every state is recurrent.

A return to zero is possible only at even times, andStirling's estimate makes this asymptotic to a positive constant timesA state is recurrent exactly when the sum of its return probabilities diverges. If $p=1/2$, the resulting $k^{-1/2}$ series diverges. If $p\ne 1/2$, then $4p(1-p)<1$ and the series converges geometrically. Thus the walk is recurrent exactly when $p=1/2$.