Past exam of the options II course of the University of Oxford / 2009 / M / M5

For $i\notin A$, the event of ever reaching $A$ can be conditioned on the first step. Given $X_1=j$, its remaining hitting probability is $h_j$ by the Markov property, including $h_j=1$ if $j\in A$. The law of total probability thus givesFor $i\in A$ the definition uses time zero and gives $h_i=1$, so the harmonic equation is needed only outside $A$.

First assume $0<q<p<1$. Stop the nearest-neighbour walk on reaching zero or a finite upper level $L>i$. Its probability $h_i^{(L)}$ of reaching zero first solves $h_i^{(L)}=p{h}_{i+1}^{(L)}+q{h}_{i-1}^{(L)}$, with boundary values one and zero. Solving this second-order recurrence givesAbsorption in the finite interval occurs almost surely: from every interior state there is a uniformly positive probability of an absorbing run within $L$ steps. As $L\to \infty$, the events of hitting zero before $L$ increase to the event of ever hitting zero, since a path reaching zero at a finite time has a finite maximum before then. ConsequentlyThis limiting argument selects the actual hitting probability, rather than just an arbitrary bounded solution of the infinite recurrence. If $q=0$, the walk moves deterministically upwards from positive states and the probability is zero there; from state zero it is one. These agree with the formula interpreted separately at $i=0$.

After a first step to $+1$, the walk moves away from zero with probability $3/4$ and towards it with probability $1/4$, so the previous result gives return probability $1/3$. After a first step to $-1$, reflect the line: movement away from zero has probability $2/3$ and towards zero $1/3$, giving return probability $1/2$. Weighting by the first-step probabilities gives

All entries must lie between zero and one. The tightest restriction comes from $p_{01}=\lambda$, so together with the given positivity the parameter range is

Every finite upward path has positive probability, since each upward transition probability is positive. Every state $i\ge 1$ has a positive transition probability to zero because $\lambda /(i+1)<1$. From zero one can therefore reach every state, and from any state return to zero and then reach any other. Hence the chain is irreducible.

If $\lambda <1$, state zero has a positive self-loop, so its period is one. If $\lambda =1$, that self-loop disappears, but the paths $0\to 1\to 0$ and $0\to 1\to 2\to 0$ have positive probabilities and lengths two and three. Their greatest common divisor is one. Thus zero has period one in this boundary case too; irreducibility makes the whole chain aperiodic.

For $j\ge 1$, the only incoming transition is from $j-1$, so a stationary distribution must satisfyNormalization gives ${\pi }_0{e}^{\lambda }=1$, henceThe zero-state balance also holds: ${\sum }_i{\pi }_i(1-\lambda /(i+1))=1-{\sum }_{j\ge 1}{\pi }_j={\pi }_0$. Thus the Poisson distribution is indeed stationary, including when $\lambda =1$. Irreducibility gives uniqueness.