= Solution
The modified <transition matrix> $\widetilde P$ describes the original <Markov chain> until it hits $B$, and then keeps it at the first state hit. Each state of $B$ is therefore an <absorbing state>, and
$$
(\widetilde P^{\,n}\mathbf1_B)_i=\mathbb P_i(\tau\le n).
$$
For a <nonnegative superharmonic function for a Markov chain> $g$ satisfying the boundary condition, part (c) and positivity of the <transition matrix> give by induction $g\ge\widetilde P^{\,n}g$. Also $g\ge\mathbf1_B$. Consequently
$$
g_i\ge(\widetilde P^{\,n}g)_i\ge(\widetilde P^{\,n}\mathbf1_B)_i=\mathbb P_i(\tau\le n).
$$
Let $n\to\infty$ to obtain $g_i\ge h_i$. Since $h$ satisfies the conditions by parts (a),(b), \b[$h$ is their pointwise minimal solution]. This extends the <hitting probability is the minimal nonnegative harmonic extension> principle to the superharmonic inequality. Nonnegative sums justify the argument on a countable state space as well as a finite one.
Now suppose the <Markov chain> is irreducible and recurrent, and let $g$ satisfy condition (a). In such a chain every state is hit almost surely from every other state. To see this directly, recurrence gives infinitely many visits to a starting state $i$. Irreducibility supplies a finite path from $i$ to any target $j$ with positive probability $\alpha$, chosen with no return to $i$ before reaching $j$. At each successive return to $i$, the <Strong Markov property> gives another trial with conditional success probability at least $\alpha$. The chance of avoiding $j$ through $N$ such trials is at most $(1-\alpha)^N$, which tends to zero.
Apply the absorbing modification at the singleton $\{j\}$. Part (c) holds for arbitrary $g$ satisfying (a), so
$$
g_i\ge(\widetilde P^{\,n}g)_i\ge g_j\mathbb P_i(\tau_j\le n).
$$
Since $g_j\ge0$ and the probability tends to one, $g_i\ge g_j$. Interchanging $i,j$ yields equality. Thus \b[every nonnegative superharmonic function is constant] on an irreducible recurrent <Markov chain>.
Conversely, if an irreducible <Markov chain> is not recurrent, choose a state $j$ with return probability $r_j=\mathbb P_j(\tau_j^+<\infty)<1$. Its <hitting probability> $h$ for $\{j\}$ satisfies condition (a), and $h_j=1$. The first-step formula for a return after time zero gives $(Ph)_j=r_j<1$. Therefore $h$ cannot be constant: a constant function with value one at $j$ would have $(Ph)_j=1$. This supplies a nonconstant nonnegative superharmonic solution. Hence
$$
\boxed{P\text{ is recurrent}\iff\text{every solution of (a) is constant}.}
$$
The admissible constant solutions are precisely the nonnegative constants. This is <recurrence characterized by constant nonnegative superharmonic functions>.
Back to article page
