Past exam of the mathematics course of the University of Cambridge / 2017 / ii / Paper 1 / 27K / a / Solution

A time-homogeneous continuous-time Markov chain on a countable state space is a stochastic process with the Markov property: conditional on the current state, the law of its future depends on neither its past nor the current time. In the usual conservative, nonexplosive setting its sample paths are right-continuous step functions. Its Q-matrix $Q=(q_{ij})$ has $q_{ij}\ge 0$ for $j\ne i$, $q_i=-q_{ii}={\sum }_{j\ne i}{q}_{ij}<\infty$, and infinitesimal transition probabilities ${\mathbb{P}}_i(X(h)=j)=q_{ij}h+o(h)$ for $j\ne i$.

On entering $i$, the holding time has distribution $\mathrm{Exp}(q_i)$; independently, the next state is $j$ with probability $q_{ij}/q_i$. The successive states form the jump chain, whose transition probabilities areIf $q_i=0$, the state is absorbing, its holding time is infinite, and the jump-chain convention is $p_{ii}=1$. Nonexplosion ensures this construction defines the process for all finite times rather than accumulating infinitely many jumps in finite time.