= Solution
For a <Markov chain invariant measure>, $xP=x$, and hence $xP^n=x$. The <irreducible Markov chain> property supplies, for each $j$, an $n$ with $(P^n)_{kj}>0$. Nonnegativity gives $x_j\ge x_k(P^n)_{kj}>0$. \b[Every coordinate of a nonzero invariant measure is positive.]
Start the <Markov chain> at $k$, and let $T_k=\inf\{n\ge1:X_n=k\}$ be its first return time. Each product in the series is the probability of a path with no intermediate visit to $k$. Thus
$$
\gamma_j^k=\sum_{n\ge1}\mathbb P_k(X_n=j,\ T_k\ge n)
=\mathbb E_k\left[\sum_{n=1}^{T_k}\mathbf1_{\{X_n=j\}}\right].
$$
This is the expected number of visits to $j$ in an excursion, including the return endpoint and excluding the starting point. Because the chain is <recurrent Markov chain>, $T_k<\infty$ almost surely, and exactly one visit to $k$ is counted: \b[$\gamma_k^k=1$]. Counting the starting point instead of the return endpoint gives the equivalent usual <excursion occupation measure>.
For the lower bound, let $J=I\setminus\{k\}$, $Q=(p_{ij})_{i,j\in J}$, and $r=(p_{kj})_{j\in J}$. The stationarity equations on $J$, with $x_k=1$, are $x_J=r+x_JQ$. Iterating and using nonnegativity yields
$$
x_J=r+rQ+\cdots+rQ^{N-1}+x_JQ^N\ \ge\ \sum_{m=0}^{N-1}rQ^m.
$$
The $j$th coordinate on the right is precisely the partial excursion series for $j\ne k$. Taking its increasing limit gives \b[$x_j\ge\gamma_j^k$]; for $j=k$ equality follows from $x_k=\gamma_k^k=1$. The iteration is valid also on a countable state space because all summands are nonnegative.
Back to article page
