Continuous-time Markov chain Created 2026-09-24 Updated 2026-11-05
A Continuous-time Markov chain on a finite or countable state space is specified by transition rates for , with finite total exit rate . At a state with , it waits an exponential time of rate and then moves to with probability . A state with is absorbing: it has infinite holding time rather than a zero-rate exponential waiting law.
Its transition-rate matrix is , with off-diagonal entries and diagonal entries , so each row sums to zero. The rates describe first-order transition probabilities: for . On a finite state space, the transition semigroup is and explosion is impossible. On a countably infinite state space, finite individual exit rates do not alone exclude infinitely many jumps in finite time; nonexplosion must be checked separately.
For two states with rates and , the mean holding times are and . Detailed balance gives Stationary distribution , and
This starts at one and relaxes to the stationary mass of state zero. Changing holding rates can preserve an embedded jump chain while changing the proportions of elapsed time spent in its states.
Equilibrium distribution of a continuous-time Markov chain Created 2026-10-14 Updated 2026-11-05
An Equilibrium distribution of a continuous-time Markov chain is a probability row vector invariant under its transition semigroup:
It satisfies when the generator sums are defined. An invariant law is a property of the entire evolution, so formal generator equations must be interpreted with the process's nonexplosion and domain conditions.
For a finite irreducible chain, the equilibrium distribution is unique and every transition row converges to it. For a general finite chain, the limiting row from state is a mixture of the stationary distributions of its closed irreducible classes:
The chain eventually enters one of these classes and then mixes within it. If every closed class is a single absorbing state, this reduces to absorption probabilities at those states. Merely having some absorbing states does not exclude other closed classes.
Holding times explain the link to an embedded jump chain. For a finite irreducible chain with strictly positive exit rates and jump-chain Stationary distribution , the time-stationary distribution is
Visit frequency is multiplied by mean time per visit. In the countable setting, these time weights need a finite positive normalizer and a nonexplosive process before they define an equilibrium probability. For an alternating two-state jump chain, and rates give : slower holding rates increase time occupancy.
Invariant distribution of a continuous-time Markov chain Created 2026-09-24 Updated 2026-11-05
For a finite-state chain with transition semigroup , a probability row vector is invariant when for every . Equivalently,
Invariant distribution of an explosive chain Created 2026-09-24 Updated 2026-11-05
For an explosive chain, a normalized solution of the formal balance equations need not be an invariant probability for the minimal transition semigroup. Explosion sends Probability mass to a cemetery state, and the generator equations on the original states do not include that boundary loss.
A concrete example uses the reflected jump chain on which moves from zero to one, and from moves up with probability and down with probability . Give state holding rate . The embedded chain has invariant weights , for . Dividing by the holding rates and normalizing yields
These weights even satisfy formal Detailed balance: , and for both opposing edge flows equal . Thus holds at each state.
Nevertheless the process explodes. Starting from zero, the embedded walk has expected visit count two at zero and three at every positive level. The zero-state return probability is ; at a positive state it is , giving those geometric visit means. Therefore the expected total elapsed holding time is
The lifetime is finite almost surely. The same bounded expected-visit argument works from every initial state, so the killed minimal semigroup loses all its mass as time tends to infinity. The displayed is consequently a formal generator-balance solution, not a stationary probability of that killed evolution. A conservative continuation after explosion would require an extra boundary rule.
Markov semigroup 2026-10-14
The transition operators of a time-homogeneous Markov process satisfy by the Markov property. They preserve constants and positivity. When the generator and function domain permit differentiation, is the Kolmogorov backward equation. Transition semigroup of a continuous-time Markov chain is the discrete-state special case.
For the finite state space , the generator or Q-matrix is defined by
Its off-diagonal entries are nonnegative jump rates and . The Transition semigroup of a continuous-time Markov chain is .
An invariant distribution of a continuous-time Markov chain is a probability row vector satisfying for every . Differentiation at zero gives ; conversely, implies . Thus
Invariant distributions need not be unique: each closed communicating class supports one, and their convex combinations are invariant. A finite irreducible chain has a unique invariant distribution.
Let be the arrival times of the independent rate- Poisson process. The increments are independent exponential variables of rate . Conditional on , the Markov property, time homogeneity, and independence of the next increment give
Hence is a discrete-time Markov chain. Its transition matrix is the Poisson-sampled continuous-time Markov chain resolvent
If , then and integration gives . Conversely, if a probability row vector satisfies , multiplying
on the right by yields . Therefore the sampled chain and the original chain have exactly the same invariant distributions.
A Q-matrix satisfies for and ; on a countable state space the exit rate is required to be finite. For a finite-state continuous-time Markov chain with transition semigroup ,
The backward and forward Kolmogorov equations are respectively
Let denote the Transition semigroup of a continuous-time Markov chain. The first person is observed in at day zero, at day seven, and at day fourteen, so the contribution is
For the second person, death is observed exactly at day six but the infection time is latent. The density of an transition at day six is
Thus the combined contribution is a function of . This illustrates how panel observations contribute transition probabilities while an exactly observed transition contributes a state probability times its transition intensity.