Alternating arrival and service queue Created 2026-10-14 Updated 2026-11-05
For positive , a continuous-time Markov chain may alternate between an arrival phase , with upward transitions at rate , and a service phase , with downward transitions at rate when the level is positive. Switching rates and are . Its stationary balance equations give a geometric invariant tail with ratio . A stationary probability exists exactly when , consistent with the stationary average service rate exceeding the arrival rate .
Consider unbound U, bound closed C and open O in a continuous-time Markov chain, with rates , , , and O absorbing over the observation interval. The mean first-opening times satisfy and , hence . Repeated unsuccessful binding attempts make this different from . A FRET trace that cannot distinguish U from C does not by itself identify the individual binding and opening rates.
Chemical master equation Created 2026-09-28 Updated 2026-11-05
For reactions with stoichiometric vectors and reaction propensity functions , the probability mass function obeys
It is the forward operator of a Markov jump process equation for the copy-number continuous-time Markov chain.
Closed migration process 2026-10-14
A closed migration process is a continuous-time Markov chain of populations moving between colonies with a conserved total number of individuals. A colony containing individuals has departure rate , with , and each departure chooses its next colony using a fixed routing matrix. Its colony populations can represent stages of a service system, including an idle stage.
Continuous-time Markov chain Created 2026-09-24 Updated 2026-11-05
A Continuous-time Markov chain on a finite or countable state space is specified by transition rates for , with finite total exit rate . At a state with , it waits an exponential time of rate and then moves to with probability . A state with is absorbing: it has infinite holding time rather than a zero-rate exponential waiting law.
Its transition-rate matrix is , with off-diagonal entries and diagonal entries , so each row sums to zero. The rates describe first-order transition probabilities: for . On a finite state space, the transition semigroup is and explosion is impossible. On a countably infinite state space, finite individual exit rates do not alone exclude infinitely many jumps in finite time; nonexplosion must be checked separately.
For two states with rates and , the mean holding times are and . Detailed balance gives Stationary distribution , and
This starts at one and relaxes to the stationary mass of state zero. Changing holding rates can preserve an embedded jump chain while changing the proportions of elapsed time spent in its states.
Detailed balance for a continuous-time Markov chain Created 2026-10-03 Updated 2026-11-05
For a continuous-time Markov chain with transition rates , Detailed balance is
for every pair of distinct states. A probability distribution satisfying these equations is an invariant distribution of a continuous-time Markov chain, because summing over incoming rates gives . Under the standard nonexplosion assumptions, the equilibrium chain is reversible.
Entropy production rate of a Markov chain Created 2026-09-28 Updated 2026-11-05
For a finite Continuous-time Markov chain, let be the rate from to , and let be its current state probability. Define forward traffic and current . The total entropy production rate, in dimensionless entropy units with natural logarithms, is
The factor avoids counting an unordered pair twice. The expression is the same as after pairing opposite directions.[ref]
Nonnegativity is elementary: for , the signs of and agree. A pair with both flows zero contributes zero. If exactly one flow is positive, its contribution is , the corresponding one-sided limiting convention. The rate vanishes exactly when every pair has equal opposing traffic at that instant, which is Detailed balance for that Probability distribution.
For a stationary three-state cycle with clockwise rate and anticlockwise rate , the stationary law is uniform. Each edge has current , so . It is zero at equal rates and positive otherwise, even though the state probabilities no longer change. Stationarity balances total flow at each state; it need not eliminate circulating currents.
Entry count of a continuous-time Markov chain Created 2026-10-05 Updated 2026-11-05
The entry count counts jumps of a continuous-time Markov chain into state by time , excluding initial occupation. It is a counting process. For a finite-state chain with homogeneous transition intensities, its mean has derivative conditional on initial state , so its expected value is the sum of incoming rates times the expected occupation times of a continuous-time Markov chain. This density is an expected count rate, not necessarily a normalized probability density function.
Expected finite-horizon occupation time Created 2026-10-14 Updated 2026-11-05
For a finite-state homogeneous continuous-time Markov chain with transition intensity matrix , integrate the indicator of occupying state over . Interchanging the bounded time integral and expected value gives the displayed integral of transition probabilities. It counts all visits, not only the first holding time. Summing over states gives . A zero eigenvalue of does not cause a problem for this integral, although it prevents casually writing it with an inverse matrix .
Explosion of a continuous-time Markov chain Created 2026-09-24 Updated 2026-11-05
Explosion occurs when infinitely many jumps take place in finite time.
Explosion is determined by whether the successive holding times of the continuous-time Markov chain have a finite sum.
Finite state space 2026-10-25
A finite set of possible complete states of a system. Its probability distribution is a finite vector. A continuous-time Markov chain on such a space has a matrix generator with nonnegative off-diagonal rates and zero column sums in the column-vector convention.
First-passage-time density Created 2026-09-29 Updated 2026-11-05
When a first-passage time has an absolutely continuous distribution, its density is , where is its survival function. In a continuous-time Markov chain, this density is the total Probability flux from surviving states into the absorbing set.
Geometric return count before absorption Created 2026-10-14 Updated 2026-11-05
Let be a state outside an absorbing set of a Markov chain. Write for its first hitting time of and . Assume each excursion ends: , and put . Then the number of visits to before absorption, including the initial visit, has the positive-support geometric distribution
After a return, the Strong Markov property restarts the same experiment. Thus and .
The excursion-ending assumption matters: on an infinite state space, an excursion may escape forever without either absorption or return. In that case the geometric parameter for the number of visits is , which need not equal the absorption-before-return probability.
For a continuous-time Markov chain, use the embedded jump chain and assume independent holding times at with exponential distribution of rate , independent of the jump destinations. The total occupation time before absorption is a geometric sum of exponential variables, with exponential distribution of rate and expected value .
Gillespie algorithm Created 2026-09-28 Updated 2026-11-05
The Gillespie algorithm samples statistically exact trajectories of a well-mixed chemical reaction network modeled by a continuous-time Markov chain. At copy-number state , each reaction propensity function is a nonnegative event rate and each stoichiometric vector specifies the update. Assume the rates have no explicit time dependence. Put . If , no further chemical reaction can occur: the state is an absorbing state.
For , sample a waiting time from the exponential distribution with rate , independently choose channel with probability , and update . Recompute the reaction propensity functions and repeat. The no-event probability is because ; multiplying by gives the joint waiting-time/channel probability density function . This factorization explains the two draws.[ref]
For with reaction propensity function , the waiting time has mean , and is an absorbing state. Exactness concerns the specified chemical master equation; spatial mixing errors and explicitly time-varying rates require a different model or sampling rule.
A queue with a Poisson process of arrivals and two independent service times with exponential distributions of different rates must distinguish the two possible single-customer states. If an arrival chooses either idle server with probability , the continuous-time Markov chain is reversible. Write for the states with only server one or two busy, and for total population. Its detailed balance equations give , and . These weights normalize exactly when . In equilibrium, reversed upward transitions have total rate in every state, so time reversal of a continuous-time Markov chain makes the departure stream a Poisson process of rate . This output conclusion requires stationarity.
The holding rate at state of a continuous-time Markov chain with generator matrix is
For a conservative chain with finite , its holding time has exponential distribution: and . Conditional on a jump, the next state is with probability . Thus the rate determines when departure occurs, while normalized outgoing rates determine where it goes.
If the outgoing rates are and for two distinct other states, the total rate is five, the mean holding time is , and departure chooses them with probabilities and . A zero rate gives an absorbing state with infinite holding time. Statewise finite rates alone do not rule out explosion of a continuous-time Markov chain.
Holding time Created 2026-09-29 Updated 2026-11-05
The holding time of a continuous-time Markov chain is the time spent in its current state before the next jump. For it has an exponential distribution of total outgoing rate ; at an absorbing state with it is infinite.
Infinitesimal generator (stochastic processes) Created 2026-09-24 Updated 2026-11-05
The Infinitesimal generator describes the instantaneous action of a Markov transition semigroup on functions of the state. For a strongly continuous semigroup on a specified Banach space of functions, its generator is
The domain and the limit topology are part of the definition; different function spaces can give different operator domains. Pointwise generator formulas are often computed first on smooth test functions.
For a continuous-time Markov chain with rates ,
For Brownian diffusion with variance , on smooth compactly supported functions. For a deterministic flow , the chain rule instead gives . These examples turn transition rules or local motion into an operator.
For suitable and integrability conditions, Dynkin's formula makes a martingale. Taking expectations yields evolution equations; stopping at a boundary gives equations such as for hitting probabilities and for mean first-passage times. The generator is useful because local drift, diffusion or jump rates encode global transition and boundary questions.
Interchange process 2026-10-06
Place one distinct label at each vertex of a finite connected weighted graph. At rate , interchange the labels at and . This continuous-time Markov chain has a uniform distribution on a finite set as its stationary distribution. The position of one label is the random walk with the same rates.
For a continuous-time Markov chain with strictly positive finite holding rates whose jump times tend to infinity, the chain and its jump chain have the same directed paths of positive-probability jumps. They are therefore irreducible simultaneously, and a state is recurrent for one exactly when it is recurrent for the other: the event of returning after departure depends only on the sequence of visited states.