Stationary distribution Created 2026-09-24 Updated 2026-10-14
A stationary distribution is a probability distribution preserved by a Markov chain's evolution. For a Discrete-time Markov chain in the row convention,Thus is a normalized nonnegative left eigenvector of the transition matrix for eigenvalue one. Starting with distributed as makes every have that same probability distribution. For a time-homogeneous Markov chain, the path probabilities are also unchanged by shifting all observation times together: the process has strict stationarity. This does not mean its individual paths are constant. See Hao Wu's MIT lecture 2, stationary distributions.
For the sunny/rainy transition matrix, write . The sunny component of gives , so andMoreover,These equal stationary flows verify detailed balance and show that the Markov chain is reversible. The diagonal detailed balance equations hold automatically. Only of the stationary days are rainy, but of the days following a rainy day are rainy: stationarity permits dependence.
A finite Markov chain that is an irreducible Markov chain has a unique stationary distribution. If it is also an aperiodic Markov chain, for every initial probability distribution. The sunny/rainy example satisfies both conditions: both directions are possible and each state has a positive holding probability of a Markov chain. See Hao Wu's MIT lecture 4, convergence theorem. For an infinite state space, the additional positive recurrence hypothesis in the countable-state Markov chain convergence theorem matters.
Existence of a stationary distribution alone does not give this convergence. Consider the deterministic two-cycleHere , but starting in the first state gives alternating probability distributions . Its period of a state in a Markov chain is two. The running fraction of visits to either state nevertheless tends to , illustrating the Markov chain ergodic theorem. Starting with instead keeps the one-time probability distributions stationary even though every path alternates.
Strong lumpability Created 2026-10-06 Updated 2026-11-05
A partition of a Markov chain state space is strongly lumpable when, for every target block , the sum is identical for all source states in a given source block. These sums define the transition matrix of the block-valued Markov chain, independently of the initial law. Failure of this condition means the projection is not guaranteed to be a homogeneous Markov chain for every initial law, although special initial laws may still produce a Markov process.
Strong stationary time Created 2026-09-24 Updated 2026-11-05
A randomized stopping time is a strong stationary time for a Markov chain with stationary distribution when has distribution and is independent of . Equivalently,
Subsampled Markov chain 2026-10-31
Let be a time-homogeneous Markov chain with transition matrix , and let be a positive integer. Then is a Markov chain with transition matrix and the same initial distribution. The Markov property and the Chapman-Kolmogorov equation give . Conditioning again on the smaller history leaves the right-hand side unchanged, since it is determined by . This proves the Markov property of the sampled sequence, rather than only identifying its marginal transition probabilities. Periodicity of the original chain may split the sampled chain into several closed classes.
Terminal model of copolymerization 2026-11-05
The terminal model of copolymerization represents a growing copolymer chain's addition kinetics using only the identity of its final monomeric unit. For two incoming monomers, there are two end types and four chain-growth propagation rate constants , where identifies the existing end and the incoming monomer. Earlier sequence history is omitted from this kinetic state description.[ref]
Radical Copolymerization Reactivity Ratios by IUPAC. Published technical Introduction, terminal-model assumption
At a fixed reacting-mixture composition, addition rates give a two-state Markov chain. With the monomer mole fractions and , the monomer reactivity ratios, its transition matrix isRows label the current terminal unit; columns label the newly added unit. Each row sums to one. This interprets a sequence at fixed feed and neglects initiation, termination and feed drift over the sampled run.
For an original example, take and . Then and both cross-addition probabilities are . The stationary distribution is , so the long-chain mean composition is equimolar, while equal consecutive units are favored. A run of type 1 has a geometric distribution with stopping probability and mean length 3. Independent equimolar placement would instead give mean run length 2. Mean composition alone does not determine sequence correlations.
The copolymer composition equation concerns the instantaneous incorporated composition implied by this model. Applying it over a finite conversion interval requires following changes in the reacting mixture. The IUPAC recommendation explicitly bases ratio estimation on conversion and composition observations at different starting feeds.[ref]
Radical Copolymerization Reactivity Ratios by IUPAC. Official recommendation announcement, measurement design
Historically, Frank R. Mayo and Frederick M. Lewis published a 1944 theoretical comparison of monomers with a study of styrene and methyl methacrylate.[ref]
Copolymerization. I. A Basis for Comparing the Behavior of Monomers in Copolymerization; The Copolymerization of Styrene and Methyl Methacrylate by Frank R. Mayo and Frederick M. Lewis. Publisher title, byline and publication record
A successful terminal-model fit is evidence about a chosen kinetic representation, not proof that every chemical influence vanishes beyond the last unit. Thinning of a Markov chain 2026-10-05
Thinning keeps every th state of a Markov chain. The retained chain has transition kernel and the same stationary distribution. It is generally still dependent, and reducing correlation between retained draws does not by itself improve precision per original transition. Keep all post-warmup draws unless storage or later processing requires thinning; assess precision using the effective sample size of a Markov chain.
Time-reversed Markov chain Created 2026-09-29 Updated 2026-11-05
For a stationary Markov chain with transition matrix and Stationary distribution , the time-reversed Markov chain has transition matrixon the support . The Stationary distribution is essential: Bayes' rule gives . The quotient is not defined at a zero-mass state, so arbitrary states outside the stationary support must not be included without an extra convention.
The stationarity equations give . Also , so the reversed kernel has the same invariant law. Multiplying the pairwise identity along a finite path proves equality between its forward Probability and the Probability of the reversed path under .
The reversed chain need not be the original chain. A directed three-state cycle has uniform Stationary distribution; time reversal traverses the cycle in the opposite direction. A stationary chain is reversible exactly when , which is equivalent to Detailed balance.
Transient state Created 2026-09-24 Updated 2026-11-05
Transition-completion time in a Markov chain Created 2026-10-07 Updated 2026-11-05
For a Markov chain with transition matrix , fix a directed transition with . Its transition-completion time isIt counts the successful final step. Merely visiting by another route does not complete this event. In a finite irreducible chain its mean is finite and first-step analysis givesThe subtracted continuation term removes exactly the case where the designated transition has just succeeded. An equivalent construction sends that transition to a new absorbing success state and computes its Expected hitting time.
For the two-state chain , , with , the waiting time for has means and . Starting at one, the hitting time of state one is zero, whereas the transition-completion time must first wait for and then . This example makes the difference between an edge event and a state event explicit.
Transition matrix for a subshift 2026-10-14
The adjacency matrix of a directed graph whose vertices are symbols and whose directed edges are the permitted consecutive pairs. Its entries indicate permission. In comparison, the transition matrix of a Markov chain records transition probabilities.
Transition probability Created 2026-09-29 Updated 2026-11-05
The Transition probability is the Conditional probability that a Markov chain moves to state on its next step given that its current state is .
Take states for , with deterministic and transitions with probability , with probability , with probability , and with probability . The Markov chain is transient for , null recurrent for , and positive recurrent for . To see the Markov chain recurrence threshold, eliminate the primed-state hitting probabilities to obtain , with characteristic roots and . Boundedness forces return with probability one when the second root is at least one. When it is below one, the minimal nonnegative solution gives return probability from . In the positive recurrent case, put and . The stationary distribution is , and . It follows from balance across each level cut and sums to one; at criticality the invariant weights cannot be normalized.
Two-phase inspection model 2026-10-31
A two-phase inspection model alternates between inspecting every item and inspecting each item with probability , where . Suppose item defects are independent with probability . The first phase ends after good items have been accumulated, while the second ends upon detecting a defect. With state equal to the cumulative number of good items found during the current full-inspection phase, the Markov chain has transitions , , , .
Its stationary distribution is for and . Indeed, balance gives equal mass to every phase-A state and ; normalization fixes the common mass. The finite chain is irreducible and aperiodic, so the Markov chain ergodic theorem identifies these probabilities with long-run occupation fractions. The long-run inspected fraction is . The fraction of all production consisting of undetected defective items is .
An important modeling choice is whether defects during full inspection preserve accumulated good items or reset the count. Resetting produces the distinct consecutive-clearance inspection model and changes the long-run fractions.
Uniform depletion and refill Markov chain 2026-10-31
Refill the stock to a fixed positive size immediately whenever a step empties it. The resulting Markov chain has state space and transitions for and . It is irreducible and aperiodic, since every state reaches , reaches every state, and . Its unique stationary distribution is and for . Normalization is the harmonic number identity. For , stationarity follows from ; the same telescoping sum starting at gives stationarity at . The finite-state convergence theorem therefore makes this stationary distribution the limiting distribution from every initial state.
Uniform depletion Markov chain 2026-10-31
A finite stock decreases by choosing a uniformly random positive number of its remaining items at every step. The remaining stock is a Markov chain with for and . State is an absorbing state, and every positive state is a transient state. If denotes the expected number of positive-stock steps before absorption, a first-step decomposition gives and . Subtracting consecutive equations yields , so , the th harmonic number.
Weighted graph Created 2026-09-24 Updated 2026-11-05
An edge-weighted graph assigns a numerical weight to each graph edge. The meaning depends on the model: a weight can be a length, a cost, or an electrical conductance. Arbitrary weights do not automatically define transition probabilities.
For the random walk convention used here, take a locally finite undirected graph with symmetric nonnegative electrical conductances , and optional nonnegative killing weights . AssumeThe Markov chain moves from to with probability and goes to a cemetery state with probability . These probabilities sum to one. With no killing, detailed balance holds with respect to the measure ; normalization gives a stationary distribution when its total mass is finite.
For example, electrical conductances and on the two edges incident to an graph vertex give outgoing probabilities and . Treating the same weights as electrical resistances would require reciprocal electrical conductances and reverse these preferences. The distinction is essential in the electrical network interpretation.
Yuval Peres Created 2026-10-13 Updated 2026-11-05
Yuval Peres coauthored a treatment of Markov chains and mixing times. Coupling, spectral methods and probabilistic estimates quantify how quickly a chain approaches equilibrium, connecting a limiting distribution with the time needed to approximate it.[ref]
Markov Chains and Mixing Times, second edition by David A. Levin and others. author attribution and the described mathematical development
Portrait of Yuval Peres
. George M. Bergman. CC BY-SA 4.0, via Wikimedia Commons. Source.
.jpg?width=250)