Past exam of the mathematics course of the University of Cambridge / 2007 / ii / Paper 3 / 25J (Applied Probability)

For a finite-state continuous-time Markov chain with generator matrix $Q$, $P(t)=e^{tQ}$. The exponential solves both Kolmogorov equations, $P^{\prime }=QP=PQ$, with initial condition $P(0)=I$.

Write $Q=1r^T-RI$, with $r=R\pi$. Then ${\pi }^{T}1=1$ andThus $\pi$ is a stationary probability vector, a left eigenvector for eigenvalue zero. If any probability vector $\nu$ is stationary, the same calculation gives ${\nu }^{T}Q=r^T-R{\nu }^T=0$, forcing $\nu =\pi$. Hence the equilibrium is unique, even if some $r_j$ vanish and the chain is not irreducible; those zero-rate states are transient or inaccessible rather than extra stationary classes.

If $x^{T}1=0$, then $x^{T}Q=-R{x}^T$. Induction gives $x^T{Q}^n=(-R{)}^n{x}^T$. Summing the matrix-exponential series yieldsThus deviations of total mass zero decay with the single rate $R$.

The vector ${\delta }_i-\pi$ has component sum zero, so part (c) applies to it. Part (b) gives ${\pi }^{T}P(t)={\pi }^T$. Decompose ${\delta }_i^T={\pi }^T+({\delta }_i^T-{\pi }^T)$ to obtainThis is the full transition-law row, showing how the initial colour survives with exponentially decreasing weight.

Reading the $j$th component givesThe chain has the interpretation of a rate-$R$ Poisson clock which independently redraws a colour from $\pi$, allowing a redraw of the same colour. No redraw by time $t$ has probability $e^{-Rt}$ and retains the initial state; at least one redraw gives distribution $\pi$.