Past exam of the mathematics course of the University of Cambridge / 2024 / ib / Paper 1 / 19H / a / Solution

Starting from $i$, the probability of reaching $j$ before returning to $i$ is $\alpha$. Conditional on reaching $j$, each visit to $j$ is followed by a hit on $i$ before the next return to $j$ with probability $\alpha$, by symmetry and the Strong Markov property. For $\alpha >0$, the number of visits after entry is therefore geometric on $\{1,2,\dots \}$ with mean $1/\alpha$. Hence the two-state excursion visit law givesIn the degenerate case $\alpha =0$, the chain never reaches $j$ before its return to $i$, so $N=0$ almost surely and ${\mathbb{E}}_{i}N=0$.