= Solution
{c}
An <absorbing state> $i$ has $p_{ii}=1$; once reached, the chain never leaves it. For a nonabsorbing initial state, condition on its first transition and use the <Markov property>:
$$
h_i=\sum_{j\in S}
\mathbb P(X_1=j\mid X_0=i)\,
\mathbb P(\text{eventually hit }1\mid X_1=j)
=\sum_jp_{ij}h_j.
$$
Here $h_1=1$ and $h_4=0$. The other equations are
$$
h_2=\frac13+\frac23h_3,\qquad h_3=\frac14h_2.
$$
Solving gives
$$
\boxed{h_2=\frac25,\qquad h_3=\frac1{10},\qquad h_4=0.}
$$
For absorption into either state one or four, let $e_i$ be the expected number of steps, with $e_1=e_4=0$. <first-step analysis> gives
$$
e_2=1+\frac23e_3,\qquad e_3=1+\frac14e_2.
$$
Hence $e_2=2$ and
$$
\boxed{e_3=\frac32.}
$$
The expectation is finite because each transient two-step cycle has <probability> at most $(2/3)(1/4)=1/6$ of remaining transient; the survival tail is geometrically bounded. This also justifies solving the expectation equations without assuming finiteness in advance.
Back to article page
