Continuous-time Markov chain
ID: continuous-time-markov-chain
A Continuous-time Markov chain on a finite or countable state space is specified by transition rates for , with finite total exit rate . At a state with , it waits an exponential time of rate and then moves to with probability . A state with is absorbing: it has infinite holding time rather than a zero-rate exponential waiting law.
Its transition-rate matrix is , with off-diagonal entries and diagonal entries , so each row sums to zero. The rates describe first-order transition probabilities: for . On a finite state space, the transition semigroup is and explosion is impossible. On a countably infinite state space, finite individual exit rates do not alone exclude infinitely many jumps in finite time; nonexplosion must be checked separately.
For two states with rates and , the mean holding times are and . Detailed balance gives Stationary distribution , andThis starts at one and relaxes to the stationary mass of state zero. Changing holding rates can preserve an embedded jump chain while changing the proportions of elapsed time spent in its states.
New to topics? Read the docs here!