Past exam of the options I course of the University of Oxford / 2010 / M / M2 / c

The chain alternates at every step between the two classes $\{1,2\}$ and $\{3,4\}$. It is irreducible and has period two: returns require an even number of steps, and a two-step return has positive probability.

Writing the off-diagonal block as $B=\left(\begin{array}{cc}1/3& 2/3\\ 2/3& 1/3\end{array}\right)$, its square is $B^2=\left(\begin{array}{cc}5/9& 4/9\\ 4/9& 5/9\end{array}\right)$. HenceEvery two steps remain within the initial class, where the sampled chain is irreducible and aperiodic.

State $2$ can be reached from state $1$ only at an even time. In the two-step sampled chain, each step while still at state $1$ reaches state $2$ with probability $4/9$, and otherwise stays at state $1$ with probability $5/9$. Thus, if $T$ is the original hitting time,Equivalently $T=2G$ where $G$ has the geometric distribution on positive integers with parameter $4/9$.

On the class $\{1,2\}$, the two-step transition matrix is symmetric with positive entries, so its stationary distribution is $(1/2,1/2)$ and its distribution converges to that law. ThereforeExplicitly the two eigenvalues are $1$ and $1/9$, giving $P(X_{2n}=2\mid X_0=1)=\frac{1}{2}(1-9^{-n})$. The original chain has probability zero of being in state $2$ at odd times, consistent with its period two.