Past exam of the probability, statistics and options I course of the University of Oxford / 2010 / M / 2 (M2)

A stationary distribution $\pi =({\pi }_1,{\pi }_2)$ satisfies $\pi P=\pi$ and ${\pi }_1+{\pi }_2=1$. The first equation reduces to the balance relation $a{\pi }_1=b{\pi }_2$. Since $a,b>0$, normalization gives the unique solution

Condition on the full sampled history through time $n$. The Markov property of $X$ implies that its future after $2n$ depends on that history only through $X_{2n}=Y_n$. Summing over the intermediate state givesThus $Y$ is a Markov chain with transition matrix $P^2$. This is a deterministic-time sampled Markov chain.

The chain alternates between the classes $\{1,2\}$ and $\{3,4\}$. It is irreducible and has period two, since no odd return is possible and each state has a positive two-step return probability. Matrix multiplication givesThe even-time sampled chain stays in its initial class and moves between its two states with probability $4/9$ at each sampled step.

Starting at one, state two can first be reached only after an even number of steps. At every even time before that hit, the chain is again at one. By the Markov property and part (i), each successive two-step block hits two with probability $4/9$, while a failure returns to one with probability $5/9$. Therefore, for the first hitting time $H$,Equivalently $H/2$ has the geometric distribution on the positive integers with success probability $4/9$.

Within the class $\{1,2\}$, the two-step matrix has eigenvalues one and $1/9$, with stationary distribution $(1/2,1/2)$. Diagonalizing this symmetric matrix, or solving the recursion $p_{n+1}=4/9+p_n/9$ from $p_0=0$, givesConsequently the limit is $1/2$. Restriction to even times is essential because the original chain has period two.