 Solution

A state $i$ is absorbing if $p_{ii}=1$, so once visited it is never left. Condition on the first step and use the Markov property: for $i\ne 1$, the subsequent probability of ever reaching state one is $h_j$ given $X_1=j$. Hence $h_i={\sum }_j{p}_{ij}{h}_j$, with $h_1=1$.

In this chain state four is a separate absorbing state, so $h_4=0$. The other two equations areSolving givesLet $t_i$ be the mean time to hit the absorbing set. From either transient state the next-step absorption probability is at least $1/3$, so its tail is geometrically bounded and the expectations are finite. First-step analysis gives $t_2=1+\frac{2}{3}{t}_3$ and $t_3=1+\frac{1}{4}{t}_2$. Thus $t_3=5/4+t_3/6$ and