= Two-state excursion visit law
Let $i\ne j$ belong to an <irreducible Markov chain>[irreducible] <recurrent Markov chain>[recurrent] <Discrete-time Markov chain>. Write $T_k=\inf\{n\geq0:X_n=k\}$ and $T_k^+=\inf\{n\geq1:X_n=k\}$. Define
$$
a=\mathbb P_i(T_j<T_i^+),\qquad
b=\mathbb P_j(T_i<T_j^+).
$$
The positive return convention is essential: $T_i$ itself is zero when starting at $i$. Let $N$ count visits to $j$ before the first positive return to $i$, including repeated holding visits at $j$. Then
$$
\mathbb P_i(N=0)=1-a,\qquad
\mathbb P_i(N=k)=ab(1-b)^{k-1}\quad(k\geq1),
\qquad \mathbb E_iN=a/b.
$$
With <Probability> $a$ the excursion reaches $j$. From each visit there, the <Strong Markov property> gives <Probability> $b$ of hitting $i$ before the next return to $j$. <recurrence> ensures there is no third possibility of escaping both states forever. Thus, conditional on reaching $j$, the visit count has a positive-support <geometric distribution> with <mean> $1/b$.
For a two-state chain which chooses either state independently with <Probability> $1/2$ at every step, $a=b=1/2$. Half of the return cycles from zero never visit one; the rest visit it a geometrically distributed number of times of <mean> two. The unconditional expected visit count is therefore one.
Back to article page