The pair is an irreducible positive recurrent product Markov chain with stationary distribution
The stationary cycle occupation formula says that, during one return cycle to a state , the expected number of visits to a set is . Take
Then
The initial and terminal states both have , so either convention for including the endpoints gives the same count. Thus