Nonnegative superharmonic function for a Markov chain ($g\ge Pg\ge 0$)

For a transition matrix $P$, a finite-valued function on its state space is nonnegative superharmonic when $g\ge Pg\ge 0$. Thus $g\ge 0$ and its expected value cannot increase after one step. Replacing transitions on a target set by identity rows preserves $g\ge \tilde{P}g$. Iterating this inequality gives $g\ge {\tilde{P}}^{n}g$, a useful bound for hitting probabilities without a separate optional-stopping theorem.

An irreducible Markov chain is recurrent exactly when every nonnegative superharmonic function for a Markov chain is constant. In the recurrent case, absorb the chain at $j$. The inequality $g_i\ge ({\tilde{P}}^{n}g{)}_i\ge g_j{\mathbb{P}}_i({\tau }_j\le n)$ and almost-sure hitting give $g_i\ge g_j$; exchange $i,j$. In the transient case, the hitting probability of a state $j$ is one there and satisfies the superharmonic inequality, but its one-step average at $j$ is the strictly smaller return probability. It is a nonconstant example.