Past exam of the mathematics course of the University of Cambridge / 2017 / ib / Paper 3 / 9H (Markov Chains)

A reversible Markov chain has a stationary law $\pi$ for which every finite path has the same distribution as its time-reversed path:For a transition matrix $P$, the equivalent detailed balance condition is ${\pi }_i{P}_{ij}={\pi }_j{P}_{ji}$ for all $i,j$. It implies $\pi P=\pi$ by summing over $i$. Multiplying the balance identities along a path shows equality of its forward and reverse probabilities when $X_0\sim \pi$; conversely, taking paths of length one gives those identities.

For the usual undirected graph with at least two vertices, a simple random walk chooses a neighbor uniformly at each step, so $P_{ij}=1/d_i$ when $i$ and $j$ are adjacent and zero otherwise, where $d_i$ is the degree of a vertex. The handshaking lemma gives ${\sum }_i{d}_i=2|E|$. Thus the stationary distribution of a graph random walk isFor adjacent $i,j$, ${\pi }_i{P}_{ij}=1/(2|E|)={\pi }_j{P}_{ji}$, and for nonadjacent pairs both sides vanish. By detailed balance, this is an invariant distribution and the walk started from it is a reversible Markov chain. Connectedness makes the finite walk an irreducible Markov chain, giving uniqueness of its invariant distribution. No aperiodicity assumption is needed for uniqueness, so a bipartite graph is allowed.

If the connected graph has a single isolated vertex, the usual neighbor-choice formula is undefined. With the natural absorbing convention $P_{11}=1$, its unique invariant distribution is ${\pi }_1=1$ and it is reversible. The degree-normalization formula presupposes at least one edge.