A stationary distribution is a probability distribution preserved by a Markov chain's evolution. For a Discrete-time Markov chain in the row convention,Thus is a normalized nonnegative left eigenvector of the transition matrix for eigenvalue one. Starting with distributed as makes every have that same probability distribution. For a time-homogeneous Markov chain, the path probabilities are also unchanged by shifting all observation times together: the process has strict stationarity. This does not mean its individual paths are constant. See Hao Wu's MIT lecture 2, stationary distributions.
For the sunny/rainy transition matrix, write . The sunny component of gives , so andMoreover,These equal stationary flows verify detailed balance and show that the Markov chain is reversible. The diagonal detailed balance equations hold automatically. Only of the stationary days are rainy, but of the days following a rainy day are rainy: stationarity permits dependence.
A finite Markov chain that is an irreducible Markov chain has a unique stationary distribution. If it is also an aperiodic Markov chain, for every initial probability distribution. The sunny/rainy example satisfies both conditions: both directions are possible and each state has a positive holding probability of a Markov chain. See Hao Wu's MIT lecture 4, convergence theorem. For an infinite state space, the additional positive recurrence hypothesis in the countable-state Markov chain convergence theorem matters.
Existence of a stationary distribution alone does not give this convergence. Consider the deterministic two-cycleHere , but starting in the first state gives alternating probability distributions . Its period of a state in a Markov chain is two. The running fraction of visits to either state nevertheless tends to , illustrating the Markov chain ergodic theorem. Starting with instead keeps the one-time probability distributions stationary even though every path alternates.
A deterministic two-cycle has a stationary law, but its one-time probabilities oscillate while its time-averaged visits converge.
Details. Place books in a row and distinguish one of them. At each step select the distinguished book with Probability and each of the other books with Probability . Swap the selected book with its left neighbour; selecting the leftmost book leaves the ordering unchanged.
The distinguished book's position is a birth-death chain. From an interior position it moves left with Probability , moves right with Probability when the book immediately to its right is selected, and otherwise holds. Missing outward moves at the endpoints become holding. Detailed balance therefore gives for . With ,This finite Truncated geometric distribution is valid even when : both numerator and denominator then have the same negative sign. The uniform case is , when every book is selected equally often. Large selection Probability for the distinguished book biases it toward the left end. The strict range of ensures positive motion in both directions and avoids division by zero in the ratio.
On states , take an interior step of or with probability each and hold with probability . At an endpoint take the inward step with probability and hold with probability . For and , the chain is irreducible and its stationary distribution has endpoint weights and interior weights .
Detailed balance proves this: the doubled inward endpoint transition requires its stationary mass to be half that of an adjacent interior point, and all interior points have equal mass. The total unnormalized weight is . For there are just two equal endpoint masses.
When , positive self-loops make the chain aperiodic, so its transition probabilities converge to this law. At every move changes parity and two-step returns are possible, giving period two and no such convergence from a fixed starting state. At every state is absorbing and uniqueness of the stationary law is lost.
An irreducible Markov chain on a finite or countably infinite state space has at most one stationary distribution. On a countably infinite state space it exists exactly when the chain is positive recurrent. In that case, for every state ,where is the first return time. The expression determines every component uniquely. The formula follows by dividing the one visit to in each return cycle by the mean recurrence time; the strong law of large numbers for successive cycles makes this occupation fraction precise.
Every irreducible finite Markov chain is positive recurrent and has exactly one stationary distribution. On an infinite state space, an irreducible null recurrent Markov chain or transient Markov chain has none. For example, the simple symmetric random walk on the integers is null recurrent. Aperiodicity affects convergence of the one-time laws, but is unnecessary for existence or uniqueness of a stationary distribution.
Every finite Markov chain has a stationary distribution, without requiring Markov chain irreducibility or aperiodicity. For a row-stochastic matrix , put and choose a column probability vector . Every lies in the probability simplex. Their arithmetic averages therefore lie in this compact set, while telescoping givesA convergent subsequence has a limit in the probability simplex. Continuity of matrix multiplication gives , so is a stationary distribution. The individual iterates need not converge: a deterministic two-state swap alternates its point masses forever.
For a discrete-time Markov chain with transition matrix , these equations, together with and , say that is a stationary distribution. Incoming probability mass equals the current mass at each state. They are global balance equations and do not require detailed balance.
Let be a Markov kernel with Stationary distribution on one coordinate. On coordinates, update coordinate with kernel while leaving the others unchanged. The product distribution is invariant: its updated coordinate still has law , and independent transition randomness leaves it independent of the untouched coordinates. Choosing the updated coordinate independently of the current configuration, with any fixed selection probabilities, forms a mixture of these invariant kernels and preserves the same product law.
For a two-state kernel with and , the stationary Probability of state one solves . If , thenUnder the product stationary distribution, the occupied-coordinate count has binomial distribution with parameters , mean and variance . In the exceptional case , the kernel is the identity and every is stationary; the displayed fraction is undefined and must not be used.
State-dependent coordinate selection can invalidate the conclusion. On two bits, let each single-coordinate kernel refresh to a fair bit, but select coordinate one whenever its current value is zero and coordinate two otherwise. Starting with independent fair bits, the Probability that the first bit equals one after a step is , so the uniform product law is not invariant. Invariance of each coordinate-update kernel alone is insufficient when the mixture weights depend on the state.
Articles by others on the same topic
A **stationary distribution** is a concept primarily used in the context of Markov chains and stochastic processes. It refers to a probability distribution that remains unchanged as time progresses. In other words, if the system is in the stationary distribution, the probabilities of being in each state do not change over time.
