= Ordered-state visit times in a reversible chain
Let a <Reversible Markov chain> start in its <Stationary distribution> $\pi$, and let $a\ne b$. Define $\tau_{ab}$ to be the first time an occurrence of $a$ has been followed strictly later by an occurrence of $b$:
$$
\tau_{ab}=\inf\{n\geq1:\text{there is }0\leq i<n\text{ with }X_i=a,\ X_n=b\}.
$$
For each fixed $n$, the <event> $\{\tau_{ab}\leq n\}$ says that the path segment $(X_0,\ldots,X_n)$ contains $a$ before $b$. Reversing that segment changes the <event> into the existence of $b$ before $a$. Stationary reversibility therefore gives $\mathbb P_\pi(\tau_{ab}\leq n)=\mathbb P_\pi(\tau_{ba}\leq n)$ for every $n$, so the two completion times have the same distribution.
For a finite <irreducible Markov chain>[irreducible] chain, all these expected <hitting times> are finite. Let $T_j=\inf\{n\geq0:X_n=j\}$ and $H_{ij}=\mathbb E_iT_j$, with $H_{jj}=0$. The <Strong Markov property> at $T_a$ gives $\mathbb E_\pi\tau_{ab}=\mathbb E_\pi T_a+H_{ab}$. Applying the same argument with $a,b$ reversed yields
$$
H_{ab}-H_{ba}=\mathbb E_\pi T_b-\mathbb E_\pi T_a.
$$
If the diagonal is instead defined by the positive <mean recurrence time>[mean return time] $\mathbb E_jT_j^+=1/\pi_j$, its contribution to each stationary-weighted <mean> is one by Kac's lemma. Those equal additions cancel in the displayed difference.
Back to article page