Past exam of the probability course of the University of Oxford / 2014 / 3 / a

Write the stationary distribution as a row vector $\pi =(a,b,c,d)$. For the first transition matrix, $\pi P=\pi$ gives $a=a/2+d$, $b=a/2+b/2$, $c=b/2$ and $d=c$. Hence $b=a$ and $c=d=a/2$. Normalization yieldsBy the stated common-stationary distribution property, this is also the answer for the second transition matrix.

The first Markov chain is a finite irreducible Markov chain: its positive-probability directed cycle reaches all four states. The self-loop at state 1 makes it an aperiodic Markov chain. Convergence to an invariant distribution therefore givesThe second chain alternates between the sets $\{1,3\}$ and $\{2,4\}$. It is a periodic Markov chain with period two, so $p_{11}^{(2m+1)}=0$. From either state 2 or state 4 the next-step probability of entering state 1 is $2/3$. Therefore $p_{11}^{(2m)}=2/3$ for every $m\ge 1$. The subsequences have different limits, so the second return-probability sequence does not converge.