= Solution
A time-homogeneous <continuous-time Markov chain> on a countable state space is a stochastic process with the <Markov property>: conditional on the current state, the law of its future depends on neither its past nor the current time. In the usual conservative, nonexplosive setting its sample paths are right-continuous step <functions>. Its <Q-matrix> $Q=(q_{ij})$ has $q_{ij}\ge0$ for $j\ne i$, $q_i=-q_{ii}=\sum_{j\ne i}q_{ij}<\infty$, and infinitesimal transition probabilities $\mathbb P_i(X(h)=j)=q_{ij}h+o(h)$ for $j\ne i$.
On entering $i$, the <holding time> has distribution $\operatorname{Exp}(q_i)$; independently, the next state is $j$ with <probability> $q_{ij}/q_i$. The successive states form the <jump chain>, whose transition probabilities are
$$
\boxed{p_{ij}=q_{ij}/q_i\quad(j\ne i),\qquad p_{ii}=0\quad(q_i>0)}.
$$
If $q_i=0$, the state is absorbing, its <holding time> is infinite, and the jump-chain convention is $p_{ii}=1$. Nonexplosion ensures this construction defines the process for all finite times rather than accumulating infinitely many jumps in finite time.
Back to article page
