= Solution
The first <Markov chain> is a finite <irreducible Markov chain>: its positive-probability directed cycle reaches all four states. The self-loop at state 1 makes it an <aperiodic Markov chain>. <Convergence to an invariant distribution> therefore gives
$$
\boxed{p_{11}^{(n)}\longrightarrow\pi_1=\frac13\quad\text{for the first matrix}.}
$$
The second chain alternates between the <sets> $\{1,3\}$ and $\{2,4\}$. It is a <periodic Markov chain> with period two, so $p_{11}^{(2m+1)}=0$. From either state 2 or state 4 the next-step <probability> of entering state 1 is $2/3$. Therefore $p_{11}^{(2m)}=2/3$ for every $m\geq1$. The subsequences have different <limits>, so \b[the second return-probability sequence does not converge].
Back to article page
