Past exam of the mathematics course of the University of Cambridge / 2024 / ib / Paper 1 / 19H (Markov Chains)

Starting from $i$, the probability of reaching $j$ before returning to $i$ is $\alpha$. Conditional on reaching $j$, each visit to $j$ is followed by a hit on $i$ before the next return to $j$ with probability $\alpha$, by symmetry and the Strong Markov property. For $\alpha >0$, the number of visits after entry is therefore geometric on $\{1,2,\dots \}$ with mean $1/\alpha$. Hence the two-state excursion visit law givesIn the degenerate case $\alpha =0$, the chain never reaches $j$ before its return to $i$, so $N=0$ almost surely and ${\mathbb{E}}_{i}N=0$.

There are no visits when the initial excursion returns to $i$ before hitting $j$, soFor $k\ge 1$, the chain must first hit $j$, return to $j$ before $i$ exactly $k-1$ times, and then hit $i$ before another return to $j$. The Strong Markov property givesThe formula also covers the boundary cases $\alpha =0$ and $\alpha =1$.

For the converse as well as the stated symmetric case, setIrreducibility and the existence of an invariant probability distribution make the chain positive recurrent, so $a,b>0$. During one return cycle from $i$ to $i$, the chain enters $j$ with probability $a$ and, after entry, makes a geometric number of visits to $j$ with success parameter $b$. ThereforeOn the other hand, the stationary cycle occupation formula givesConsequentlyIt follows thatwhich is precisely the claimed equivalence between symmetry of $i,j$ and equality of their invariant masses.