Past exam of the probability, statistics and options I course of the University of Oxford / 2009 / M / 2 / a / Solution

For $n=0$, the identity matrix gives the assertion. Suppose it holds at $n$. Conditioning on the state at time $n$ and using the Markov property givesThe nonnegative countable sum is justified by the law of total probability. Induction proves the $n$-step transition matrix is $P^n$.