Past exam of the options I course of the University of Oxford / 2010 / M / M2

Write the stationary probabilities as $({\pi }_1,{\pi }_2)$. The equation $\pi P=\pi$ reduces to $a{\pi }_1=b{\pi }_2$. Together with ${\pi }_1+{\pi }_2=1$, this givesThe positive transition probabilities make the two states communicate, so the stationary distribution is unique.

The Markov property and conditioning on the intermediate state give, for the full history through time $2n$,Condition this identity again on the smaller history $(Y_0,\dots ,Y_n)$. Its right-hand side is already determined by $Y_n=X_{2n}$, so the tower property givesThus the sampled sequence is a time-homogeneous Markov chain with transition matrix $P^2$. Checking only its one-time marginal distributions would not by itself establish the Markov property.

It is the every-second-step case of a subsampled Markov chain.

The chain alternates at every step between the two classes $\{1,2\}$ and $\{3,4\}$. It is irreducible and has period two: returns require an even number of steps, and a two-step return has positive probability.

Writing the off-diagonal block as $B=\left(\begin{array}{cc}1/3& 2/3\\ 2/3& 1/3\end{array}\right)$, its square is $B^2=\left(\begin{array}{cc}5/9& 4/9\\ 4/9& 5/9\end{array}\right)$. HenceEvery two steps remain within the initial class, where the sampled chain is irreducible and aperiodic.

State $2$ can be reached from state $1$ only at an even time. In the two-step sampled chain, each step while still at state $1$ reaches state $2$ with probability $4/9$, and otherwise stays at state $1$ with probability $5/9$. Thus, if $T$ is the original hitting time,Equivalently $T=2G$ where $G$ has the geometric distribution on positive integers with parameter $4/9$.

On the class $\{1,2\}$, the two-step transition matrix is symmetric with positive entries, so its stationary distribution is $(1/2,1/2)$ and its distribution converges to that law. ThereforeExplicitly the two eigenvalues are $1$ and $1/9$, giving $P(X_{2n}=2\mid X_0=1)=\frac{1}{2}(1-9^{-n})$. The original chain has probability zero of being in state $2$ at odd times, consistent with its period two.