= Solution
Condition on the intermediate state at time $m$. The <Markov property> and time homogeneity give
$$
p_{ij}(m+n)=\sum_k\Pr(X_m=k\mid X_0=i)\Pr(X_{m+n}=j\mid X_m=k)=\sum_kp_{ik}(m)p_{kj}(n).
$$
These nonnegative sums are valid on a countable state space. Starting from $P(0)=I$, $P(1)=P$, <induction> gives $\boxed{P(n)=P^n}$. Conditioning on the initial state now yields $\boxed{\Pr(X_n=i)=(\mu P^n)_i}$. If $\mu P=\mu$, <induction> gives $\mu P^n=\mu$ for every $n$, so all one-time distributions equal the stationary initial distribution.
Back to article page
