Past exam of the options I course of the University of Oxford / 2009 / M / M2

For $n=0$, the conditional transition probabilities are ${\delta }_{ij}$, the entries of $P^0=I$; for $n=1$ the assertion is the definition of $P$. For the induction step, condition on the state at time $n$ and use the Markov property and time-homogeneity:The countable sum is justified by the law of total probability; all terms are nonnegative. Thus the formula follows for every $n$ by induction. It is the Chapman-Kolmogorov equation for repeated one-step transitions.

Let $u_n=\mathbb{P}(X_n=1\mid X_0=1)$. Conditioning on $X_n$ givesThe fixed value is $b/(a+b)$. Subtracting it and iterating the scalar recurrence yieldsThe formula includes $n=0$. If $a+b=1$, its value at $n=0$ is interpreted directly as $u_0=1$, while $u_n=b$ for all $n\ge 1$. Since $|1-a-b|<1$, the limiting probability is $b/(a+b)$.

At the first step, the chain enters state two with probability $p$ and state four with probability $1-p$. The classes $\{2,3\}$ and $\{4,5\}$ are closed and disjoint, so only the first branch can reach state three. On $\{2,3\}$ the transition matrix is exactly the two-state matrix from part (b). Thus after the remaining $n-1$ steps, the probability of the second state in that class isFor $n=1$ it is zero, as the original matrix requires; this statement covers the zero-base exponent case without ambiguity.

State one is left after one step and never revisited. The eventual probability mass in the closed class $\{4,5\}$ isWithin this class the stationary distribution is $(b/(a+b),a/(a+b))$, because the balance equation is $a{\pi }_4=b{\pi }_5$. The two-state recurrence shows convergence from either starting state, since $|1-a-b|<1$. The other closed class never reaches state five. Therefore