 Solution

Let $T_0=0$ and let $T_{r+1}$ be the first visit to $i$ strictly after $T_r$, with $T_{r+1}=\infty$ if there is none. Put $q={\mathbb{P}}_i(T_1<\infty )$. The Strong Markov property at each finite first return time gives, by induction,The number of visits, including time zero, is $N={\sum }_{r\ge 0}{1}_{\{T_r<\infty \}}={\sum }_{n\ge 0}{1}_{\{X_n=i\}}$. By Tonelli theorem for nonnegative terms,If $q<1$, this geometric series is $1/(1-q)$, and $1-q$ is the probability of never returning. If $q=1$, every term of the final series is one and both sides are infinite. Thuswith the prescribed infinite-value convention. This covers both transient states and recurrent states.