Past exam of the options I course of the University of Oxford / 2012 / M / M1 / b / i / Solution

First the entries must be nonnegative: together with the given $a,b\ge 0$, validity of the transition matrix requires $0\le a\le 1$ and $0\le b\le 1/3$. Every row then sums to one.

The only incoming edge to state $2$ is $1\to 2$, of probability $a$. If $a=0$, no other state can reach $2$, so the chain is not irreducible. If $a>0$, all edges in the directed cyclehave positive probability, since $P_{23}=1/3+b>0$, $P_{34}=1$ and $P_{41}=1/2$. Traversing this cycle connects every state to every other state. Therefore