Past exam of the Probability, statistics and options II course of the University of Oxford / 2009 / M / M5 / a / i

For $i\notin A$, condition on the first step. The Markov property makes the chance of eventual entry after reaching $j$ equal to $h_j$, including $h_j=1$ when $j\in A$. Total probability givesThe countable sum is justified by its nonnegative terms. This first-step identity alone need not uniquely determine a hitting probability on an infinite state space; one must choose the minimal nonnegative solution or supply the appropriate boundary behavior.