Continuous-time Markov chain (source code)

= Continuous-time Markov chain
{wiki}

A <Continuous-time Markov chain> on a finite or <state space>[countable state space] is specified by <transition intensity>[transition rates] $q_{ij}\geq0$ for $i\ne j$, with finite <holding rate of a continuous-time Markov chain>[total exit rate] $q_i=\sum_{j\ne i}q_{ij}$. At a state with $q_i>0$, it waits an <exponential distribution>[exponential time] of rate $q_i$ and then moves to $j$ with <probability> $q_{ij}/q_i$. A state with $q_i=0$ is absorbing: it has infinite <holding time> rather than a zero-rate exponential waiting law.

Its <transition-rate matrix> is $Q$, with off-diagonal entries $q_{ij}$ and diagonal entries $q_{ii}=-q_i$, so each row sums to zero. The rates describe first-order <transition probabilities>: $\mathbb P(X_{t+h}=j\mid X_t=i)=q_{ij}h+o(h)$ for $j\ne i$. On a <state space>[finite state space], the <Transition semigroup of a continuous-time Markov chain>[transition semigroup] is $P_t=e^{tQ}$ and <explosion of a continuous-time Markov chain>[explosion] is impossible. On a countably infinite <state space>, finite individual <holding rate of a continuous-time Markov chain>[exit rates] do not alone exclude infinitely many jumps in finite time; <nonexplosion of a continuous-time Markov chain>[nonexplosion] must be checked separately.

For two states with rates $q_{01}=a>0$ and $q_{10}=b>0$, the <mean> <holding time>[holding times] are $1/a$ and $1/b$. <Detailed balance> gives <Stationary distribution> $(b/(a+b),a/(a+b))$, and
$$
P_{00}(t)=\frac b{a+b}+\frac a{a+b}e^{-(a+b)t}.
$$
This starts at one and relaxes to the stationary mass of state zero. Changing <holding rate of a continuous-time Markov chain>[holding rates] can preserve an <jump chain>[embedded jump chain] while changing the proportions of elapsed time spent in its states.