Past exam of the Probability, statistics and options II course of the University of Oxford / 2010 / M / M5 / a

A time-homogeneous Markov chain on a countable state space satisfies, whenever the conditioning event has positive probability,where the last probabilities do not depend on $n$. Thus the present state contains all the past information relevant to the next transition. Time homogeneity is the convention used here, since the problem specifies a single transition matrix.

The transition matrix is $P=(p_{ij}{)}_{i,j\in S}$, where $p_{ij}=P(X_{n+1}=j\mid X_n=i)$. It is row-stochastic: $p_{ij}\ge 0$ and ${\sum }_j{p}_{ij}=1$ for each $i$. Initial states of zero probability can be specified by the chain's transition mechanism, so the matrix describes the chain from any starting state.