Past exam of the Probability, statistics and options II course of the University of Oxford / 2009 / M / M5

For $i\notin A$, condition on the first step. The Markov property makes the chance of eventual entry after reaching $j$ equal to $h_j$, including $h_j=1$ when $j\in A$. Total probability givesThe countable sum is justified by its nonnegative terms. This first-step identity alone need not uniquely determine a hitting probability on an infinite state space; one must choose the minimal nonnegative solution or supply the appropriate boundary behavior.

For a finite upper barrier $L>i$, let $h_i^{(L)}$ be the probability of hitting zero before $L$. It satisfies $h_0^{(L)}=1$, $h_L^{(L)}=0$, and $h_i^{(L)}=p{h}_{i+1}^{(L)}+q{h}_{i-1}^{(L)}$. For $0<q<p$, solving the second-order recurrence with roots $1$ and $r=q/p$ givesAs $L\to \infty$, the events increase to eventual hitting of zero: any finite path reaching zero has a finite maximal height. Since $r<1$, this yields $\boxed{h_i=(q/p{)}^i}$, with $h_0=1$. In the endpoint case $q=0$, a positive initial state only moves upward and the hitting probability is zero, agreeing with the positive-state formula. The finite-barrier argument selects the correct solution rather than the also-harmonic constant one.

After the first step to $+1$, the right-hand walk moves away with probability $3/4$ and toward zero with probability $1/4$. Part (ii) therefore gives return probability $1/3$ from that side. After a first step to $-1$, reverse the sign of the coordinate: movement away from zero has probability $2/3$ and toward it probability $1/3$, giving return probability $1/2$. Weight by the probabilities of the first step from zero:There is no holding step at zero, so every positive-time return is included by this decomposition.

The necessary parameter range is $0<\lambda \le 1$: the transition probability from zero to one is $\lambda$, and this restriction also bounds all later upward transition probabilities by one.

Every upward transition has positive probability. From any positive state the transition directly to zero has positive probability, since $1-\lambda /(i+1)>0$. Thus any state can reach zero and then reach any other state by successive upward steps, proving irreducibility.

If $\lambda <1$, the holding probability at zero is positive, so its period and hence the common period of the irreducible chain is one. At $\lambda =1$ that holding probability vanishes, but zero has positive-probability return paths $0\to 1\to 0$ of length two and $0\to 1\to 2\to 0$ of length three. Their lengths have greatest common divisor one. Thus $\boxed{\text{the chain is irreducible and aperiodic for all }0<\lambda \le 1}$.

For $j\ge 1$ the only entrance is from $j-1$, so stationarity requires ${\pi }_j={\pi }_{j-1}\lambda /j$. Induction gives ${\pi }_j={\pi }_0{\lambda }^j/j!$, and normalization using the exponential series gives ${\pi }_0=e^{-\lambda }$. HenceThe remaining zero-state equation also holds: ${\sum }_i{\pi }_i\lambda /(i+1)=e^{-\lambda }(e^{\lambda }-1)=1-{\pi }_0$, so ${\sum }_i{\pi }_i(1-\lambda /(i+1))={\pi }_0$. Thus this normalized Poisson law really is stationary, not just a solution of the positive-state equations. Irreducibility makes the stationary probability law unique.