= Solution
Let $T_j=\inf\{n\geq1:X_n=j\}$. By the <Strong Markov property>, the probability of hitting $j$ and subsequently returning to $i$ is $f_{ij}f_{ji}$. This event implies a return to $i$, so \b[$f_{ii}\geq f_{ij}f_{ji}$]. For $i=j$, interpret the event as two successive returns.
Set $T_i^{(0)}=0$ and let $T_i^{(r)}$ be the $r$th successive return. Repeated use of the <Strong Markov property> gives $\mathbb P_i(T_i^{(r)}<\infty)=f_{ii}^r$. The total number $N_i$ of visits, including time zero, is both $\sum_{n\geq0}\mathbf1_{\{X_n=i\}}$ and $\sum_{r\geq0}\mathbf1_{\{T_i^{(r)}<\infty\}}$. By <Tonelli theorem>,
$$
\boxed{\sum_{n=0}^\infty\mathbb P_i(X_n=i)=\mathbb E_iN_i=\sum_{r=0}^\infty f_{ii}^r.}
$$
The identity is valid with value $+\infty$. In particular, divergence of the return-probability sum is equivalent to <Markov-chain recurrence>.
Back to article page
