Past exam of the mathematics course of the University of Cambridge / 2008 / ib / Paper 4 / 9H / Solution

The communicating classes are $T=\{1,2\}$, $B=\{3,4\}$ and $C=\{5,6,7\}$. The class $T$ is open: it can reach $B$ or $C$ and cannot be reached from either. To remain in $T$ for two steps has probability $1/6$, so ${\mathbb{P}}_i(X_{2k}\in T)=(1/6{)}^k$ for $i\in T$. Its states are transient states. All returns within $T$ have even length and a two-step return is possible, so their period of a state in a Markov chain is $2$.

The closed communicating class $B$ alternates deterministically between $3$ and $4$, and therefore consists of positive recurrent states of period $2$. The class $C$ is finite, closed and irreducible, so its states are also positive recurrent states. The self-loop at $7$ gives period $1$ there, and the period is constant on a communicating class, so every state of $C$ has period $1$.

For $C$, solve the stationary distribution equations:Hence $({\pi }_5,{\pi }_6,{\pi }_7)=(1/4,1/4,1/2)$. Finite irreducibility and aperiodicity imply convergence to this stationary distribution from any state of $C$. If $h_i$ is the probability of eventually entering $C$, the Markov property givesso $h_1=2/5$ and $h_2=3/10$. Conditioning on the finite entrance time and then letting the remaining time tend to infinity proves that the limiting probabilities in $C$ are $h_i{\pi }_j$. The probability of entrance after a large cutoff tends to zero, which justifies this conditioning limit.

For $B$, convergence depends on the parity balance on entry to a period-two Markov class. Entry always occurs at $3$. Starting from $1$, the probabilities of first entry at time $2k+1$ and at time $2k+2$ are both $\frac{1}{4}(1/6{)}^k$. Each parity therefore has total probability $3/10$, so both $p_{13}^{(n)}$ and $p_{14}^{(n)}$ tend to $3/10$, despite the recurrent class being periodic. Starting from $2$, the corresponding odd- and even-entry probabilities are $\frac{1}{2}(1/6{)}^k$ and $\frac{1}{12}(1/6{)}^k$, with totals $3/5$ and $1/10$. Consequentlyso neither limit exists. From $3$ or $4$, the probabilities at $3$ and $4$ alternate between zero and one; from $C$ they are identically zero. Finally every probability of being in $T$ tends to zero, from all starting states.

All $49$ requested conclusions are encoded below, with $\ast$ denoting a nonexistent limit:The class classification is $T$: transient, period $2$; $B$: recurrent, period $2$; $C$: recurrent, period $1$.