Past exam of the mathematics course of the University of Cambridge / 2023 / ib / Paper 1 / 19H (Markov Chains)

From any vertex, following the parent edge repeatedly reaches the root with positive probability $p^k$. From the root, any prescribed binary string $(b_1,\dots ,b_k)$ can be reached by following its bits, with positive probability $p_{b_1}\cdots p_{b_k}$. Thus every state communicates with every other state and the chain is irreducible.

At the root there is a self-loop of probability $p>0$, so the root has period one. All states of an irreducible chain have the same period; hence the chain is aperiodic.

PutThe length process $L_n=\ell (X_n)$ is a nearest-neighbour chain on the nonnegative integers: away from zero it moves up with probability $q$ and down with probability $p$, while at zero it moves up with probability $q$ and stays put with probability $p$.

Returns of the original chain to the root are exactly returns of $L_n$ to zero, so their recurrence classifications agree. The reflected biased random walk on the nonnegative integers is transient when $q>p$, null recurrent when $q=p$, and positive recurrent when $q<p$. Since $p=1-q$, the conditions are respectivelyIrreducibility transfers the classification from the root to every state.

In the positive-recurrent case, let $q=p_0+p_1<p$. Detailed balance for the length chain requiresso, with $r=q/p<1$,Normalization gives ${\pi }_0=1-r$. The self-loop at zero makes the length chain aperiodic, so the convergence theorem for irreducible positive-recurrent aperiodic Markov chains gives