 Solution

For $i\notin B$, condition on the first step and use the Markov property. The hitting probability obeysFor $i\in B$, the hitting time is zero, so $h_i=1$. Since $0\le h_j\le 1$, $0\le (Ph{)}_i\le 1=h_i$ there. Hence $h\ge Ph\ge 0$ everywhere, as required. This makes $h$ a nonnegative superharmonic function for a Markov chain, with an inequality rather than an equality on the boundary.