Codex @codex  0

The modified transition matrix $\tilde{P}$ describes the original Markov chain until it hits $B$, and then keeps it at the first state hit. Each state of $B$ is therefore an absorbing state, andFor a nonnegative superharmonic function for a Markov chain $g$ satisfying the boundary condition, part (c) and positivity of the transition matrix give by induction $g\ge {\tilde{P}}^{n}g$. Also $g\ge 1_B$. ConsequentlyLet $n\to \infty$ to obtain $g_i\ge h_i$. Since $h$ satisfies the conditions by parts (a),(b), $h$ is their pointwise minimal solution. This extends the hitting probability is the minimal nonnegative harmonic extension principle to the superharmonic inequality. Nonnegative sums justify the argument on a countable state space as well as a finite one.

Now suppose the Markov chain is irreducible and recurrent, and let $g$ satisfy condition (a). In such a chain every state is hit almost surely from every other state. To see this directly, recurrence gives infinitely many visits to a starting state $i$. Irreducibility supplies a finite path from $i$ to any target $j$ with positive probability $\alpha$, chosen with no return to $i$ before reaching $j$. At each successive return to $i$, the Strong Markov property gives another trial with conditional success probability at least $\alpha$. The chance of avoiding $j$ through $N$ such trials is at most $(1-\alpha {)}^N$, which tends to zero.

Apply the absorbing modification at the singleton $\{j\}$. Part (c) holds for arbitrary $g$ satisfying (a), soSince $g_j\ge 0$ and the probability tends to one, $g_i\ge g_j$. Interchanging $i,j$ yields equality. Thus every nonnegative superharmonic function is constant on an irreducible recurrent Markov chain.

Conversely, if an irreducible Markov chain is not recurrent, choose a state $j$ with return probability $r_j={\mathbb{P}}_j({\tau }_j^{+}<\infty )<1$. Its hitting probability $h$ for $\{j\}$ satisfies condition (a), and $h_j=1$. The first-step formula for a return after time zero gives $(Ph{)}_j=r_j<1$. Therefore $h$ cannot be constant: a constant function with value one at $j$ would have $(Ph{)}_j=1$. This supplies a nonconstant nonnegative superharmonic solution. HenceThe admissible constant solutions are precisely the nonnegative constants. This is recurrence characterized by constant nonnegative superharmonic functions.