 Solution

For $i\notin A$, the event of ever reaching $A$ can be conditioned on the first step. Given $X_1=j$, its remaining hitting probability is $h_j$ by the Markov property, including $h_j=1$ if $j\in A$. The law of total probability thus givesFor $i\in A$ the definition uses time zero and gives $h_i=1$, so the harmonic equation is needed only outside $A$.