Past exam of the mathematics course of the University of Cambridge / 2021 / ib / Paper 2 / 18H / a / ii / Solution

The renewal-reward form of Kac's lemma says that the expected number of visits to state $i$ during one return cycle from $k$ to $k$ isHere time spent means the number of discrete time instants at which the chain occupies $i$ between consecutive visits to $k$.