Past exam of the options II course of the University of Oxford / 2010 / M / M5 / a

A time-homogeneous Markov chain on a countable state space $S$ is a sequence of $S$-valued random variables for which, for every positive-probability finite history,Thus the conditional next-step law depends only on the current state, and the transition probabilities do not depend on time.

The transition matrix is $P=(P_{ij}{)}_{i,j\in S}$, where $P_{ij}=P(X_{n+1}=j\mid X_n=i)$. Its entries are nonnegative and each row sums to one. On states absent from a particular initial law, these probabilities specify the same chain when started from that state.