 Solution

The transition matrix is $P=(p_{ij}{)}_{i,j\in S}$, where $p_{ij}=P(X_{n+1}=j\mid X_n=i)$. It is row-stochastic: $p_{ij}\ge 0$ and ${\sum }_j{p}_{ij}=1$ for each $i$. Initial states of zero probability can be specified by the chain's transition mechanism, so the matrix describes the chain from any starting state.