= Solution
Let $T_0=0$ and let $T_{r+1}$ be the first visit to $i$ strictly after $T_r$, with $T_{r+1}=\infty$ if there is none. Put $q=\mathbb P_i(T_1<\infty)$. The <Strong Markov property> at each finite <first return time> gives, by induction,
$$
\mathbb P_i(T_r<\infty)=q^r.
$$
The number of visits, including time zero, is $N=\sum_{r\geq0}\mathbf1_{\{T_r<\infty\}}=\sum_{n\geq0}\mathbf1_{\{X_n=i\}}$. By <Tonelli theorem> for nonnegative terms,
$$
\sum_{n=0}^{\infty}\mathbb P_i(X_n=i)=\mathbb E_iN
=\sum_{r=0}^{\infty}q^r.
$$
If $q<1$, this <geometric series> is $1/(1-q)$, and $1-q$ is the probability of never returning. If $q=1$, every term of the final series is one and both sides are infinite. Thus
$$
\boxed{\sum_{n=0}^{\infty}\mathbb P_i(X_n=i)=
\frac1{\mathbb P_i(X_n\ne i\text{ for all }n\geq1)},}
$$
with the prescribed infinite-value convention. This covers both <transient states> and <recurrent states>.
Back to article page
