Past exam of the probability course of the University of Oxford / 2023 / 3 / a

A probability distribution on the countable state space $\mathcal{S}$ is precisely a row vector satisfyingA transition matrix is a stochastic matrix, meaningEvery row therefore gives the conditional probability distribution of the next state given the present state. These conditions do not require $\pi$ to be an invariant distribution of $P$.

The assumed identities are the detailed balance equations. Summing over $i$ and using the row sums of the transition matrix gives, for every $j$,All summands are nonnegative, so these countable sums are justified. Thusand $\pi$ is an invariant distribution.

For the countable-state Markov chain ergodic theorem, let the Markov chain be irreducible and positive recurrent, with its unique stationary distribution $\pi$. From any initial state, for every real $f$ satisfying ${\sum }_j{\pi }_j|f(j)|<\infty$,In particular, $n^{-1}{\sum }_{m=0}^{n-1}{1}_{\{X_m=j\}}\to {\pi }_j$ almost surely. This time-average conclusion requires no aperiodicity.

The countable-state Markov chain convergence theorem adds the hypothesis of an aperiodic Markov chain and concludesIt also follows that ${\mathbb{P}}_{\nu }(X_n=j)\to {\pi }_j$ for every initial probability distribution $\nu$. For an irreducible recurrent chain, the occupation-frequency theorem can also be stated as a limit $1/{\mathbb{E}}_j{T}_j^{+}$, interpreted as zero for a null recurrent state; in the positive recurrent case this equals ${\pi }_j$.