Past exam of the probability, statistics and options I course of the University of Oxford / 2007 / M / 2

An absorbing state $i$ has $p_{ii}=1$; once reached, the chain never leaves it. For a nonabsorbing initial state, condition on its first transition and use the Markov property:Here $h_1=1$ and $h_4=0$. The other equations areSolving givesFor absorption into either state one or four, let $e_i$ be the expected number of steps, with $e_1=e_4=0$. first-step analysis givesHence $e_2=2$ andThe expectation is finite because each transient two-step cycle has probability at most $(2/3)(1/4)=1/6$ of remaining transient; the survival tail is geometrically bounded. This also justifies solving the expectation equations without assuming finiteness in advance.