Solution
= Solution
For $i\notin B$, condition on the first step and use the <Markov property>. The <hitting probability> obeys
$$
h_i=\sum_jp_{ij}h_j.
$$
For $i\in B$, the <hitting time> is zero, so $h_i=1$. Since $0\le h_j\le1$, $0\le(Ph)_i\le1=h_i$ there. Hence \b[$h\ge Ph\ge0$ everywhere], as required. This makes $h$ a <nonnegative superharmonic function for a Markov chain>, with an inequality rather than an equality on the boundary.