Past exam of the probability course of the University of Oxford / 2021 / 3 / c / ii

The restriction $Q_C$ is a finite Markov chain that is an irreducible Markov chain. It is aperiodic, because its upper states have self-transition probability $a/(a+b)>0$, and the period is constant on a communicating class. We now find its stationary distribution.

Every column of $Q_C$ sums to one. Here is the full incoming-mass check, with each total measured in units of $1/(a+b)$. For $j\in L$, the incoming contributions are $a$ from $j+b\in I$ and $b$ from the self-transition. For $j\in U$, they are $b$ from $j-a\in I$ and $a$ from the self-transition.

For $j\in I$, the upward predecessor $j-a$ lies in $I$ exactly when $j\ge -c+a$, contributing $b$ units. When $j\le -c+a-1$, that predecessor is replaced by resets from all $b$ states of $L$, again contributing $b$ units. Independently, the downward predecessor $j+b$ contributes $a$ units when $j\le c-b$; when $j\ge c-b+1$, resets from the $a$ states of $U$ contribute those $a$ units instead. Thus every column total is $a+b$ units.

Therefore $Q_C$ is a doubly stochastic matrix, and the uniform distribution on $C$ is stationary:The countable-state Markov chain convergence theorem, applied to the finite closed class $C$, gives $Q_C^n(i,j)\to {\pi }_j$ for every $i\in C$. Its hypotheses hold: $C$ is finite and irreducible, hence positive recurrent, and it is aperiodic.

If $i_0>c+a$, the hitting time $H$ of $C$ has $\mathbb{P}(H>n)=(a/(a+b){)}^n$. If $i_0<-c-b$, the corresponding tail is $(b/(a+b){)}^n$. Thus $H<\infty$ almost surely from every exterior initial state. Conditioning on $H$ and $R_H$, use convergence within $C$ after entrance and the bounded convergence theorem; equivalently, first restrict to $H\le K$ and then let $K\to \infty$. This proves, for every $i_0\in \mathbb{Z}$,The limiting distribution is independent of $i_0$.