The <transition matrix> is $P=(P_{ij})_{i,j\in S}$, where $P_{ij}=P(X_{n+1}=j\mid X_n=i)$. Its entries are nonnegative and each row sums to one. On states absent from a particular initial law, these probabilities specify the same chain when started from that state.

Solution

Past exam of the options II course of the University of Oxford / 2010 / M / M5 / a / ii / Solution

 Solution (source code)