Stationary distribution Created 2026-09-24 Updated 2026-10-14
A stationary distribution is a probability distribution preserved by a Markov chain's evolution. For a Discrete-time Markov chain in the row convention,
Thus is a normalized nonnegative left eigenvector of the transition matrix for eigenvalue one. Starting with distributed as makes every have that same probability distribution. For a time-homogeneous Markov chain, the path probabilities are also unchanged by shifting all observation times together: the process has strict stationarity. This does not mean its individual paths are constant. See Hao Wu's MIT lecture 2, stationary distributions.
For the sunny/rainy transition matrix, write . The sunny component of gives , so and
Moreover,
These equal stationary flows verify detailed balance and show that the Markov chain is reversible. The diagonal detailed balance equations hold automatically. Only of the stationary days are rainy, but of the days following a rainy day are rainy: stationarity permits dependence.
A finite Markov chain that is an irreducible Markov chain has a unique stationary distribution. If it is also an aperiodic Markov chain, for every initial probability distribution. The sunny/rainy example satisfies both conditions: both directions are possible and each state has a positive holding probability of a Markov chain. See Hao Wu's MIT lecture 4, convergence theorem. For an infinite state space, the additional positive recurrence hypothesis in the countable-state Markov chain convergence theorem matters.
Existence of a stationary distribution alone does not give this convergence. Consider the deterministic two-cycle
Here , but starting in the first state gives alternating probability distributions . Its period of a state in a Markov chain is two. The running fraction of visits to either state nevertheless tends to , illustrating the Markov chain ergodic theorem. Starting with instead keeps the one-time probability distributions stationary even though every path alternates.
Figure 1.
A deterministic two-cycle has a stationary law, but its one-time probabilities oscillate while its time-averaged visits converge.
Details.
On states , take an interior step of or with probability each and hold with probability . At an endpoint take the inward step with probability and hold with probability . For and , the chain is irreducible and its stationary distribution has endpoint weights and interior weights .
Detailed balance proves this: the doubled inward endpoint transition requires its stationary mass to be half that of an adjacent interior point, and all interior points have equal mass. The total unnormalized weight is . For there are just two equal endpoint masses.
When , positive self-loops make the chain aperiodic, so its transition probabilities converge to this law. At every move changes parity and two-step returns are possible, giving period two and no such convergence from a fixed starting state. At every state is absorbing and uniqueness of the stationary law is lost.
Stationary weights of a reflected birth-death chain Created 2026-10-25 Updated 2026-11-05
Consider an irreducible Markov chain on the nonnegative integers with up-step probabilities and down-step probabilities for , reflected at zero. The displayed weights satisfy detailed balance. Every stationary distribution is proportional to them: the stationary equation at zero sets the edge flow to zero, and the remaining stationary equations make the flow constant along all edges. Hence positive recurrence is equivalent to , and then . This also allows holding probabilities of a Markov chain.
Take independent copies of the simple random walk on the integer line, each stepping left or right with equal probability at every integer time. Their product Markov chain moves all coordinates at once; each of its diagonal increments has probability . At odd times a return to the origin is impossible. At time , independence gives the displayed return probability, whose asymptotic equivalence follows from Stirling's approximation. The recurrence criterion by return probabilities and the p-series therefore imply recurrence for and a transient Markov chain for .
For the chain from the origin visits only the sublattice whose coordinates all have the same parity. This is one communicating class; restricting to it gives an irreducible Markov chain. The period of a state in a Markov chain is two here. The conclusion applies to that class, without incorrectly claiming Markov chain irreducibility on all of . This walk moves diagonally; the usual axis-step simple random walk has different transition probabilities, even though the same dimensional recurrence threshold holds.
Three-level no claims discount equilibrium Created 2026-10-14 Updated 2026-11-05
For a three-level birth-death chain whose down-or-stay claim probabilities are , with each in , detailed balance is and . Normalizing gives the displayed stationary distribution. The finite chain is irreducible and has endpoint holding probabilities, so it is aperiodic and its level proportions converge to this distribution from any initial level.
Transient Markov chain Created 2026-10-31 Updated 2026-11-05
An irreducible Markov chain is transient when its states are transient states. Starting at any one state, there is a positive Probability of never returning, and the number of visits to any fixed state is finite almost surely.
Transition-completion time in a Markov chain Created 2026-10-07 Updated 2026-11-05
For a Markov chain with transition matrix , fix a directed transition with . Its transition-completion time is
It counts the successful final step. Merely visiting by another route does not complete this event. In a finite irreducible chain its mean is finite and first-step analysis gives
The subtracted continuation term removes exactly the case where the designated transition has just succeeded. An equivalent construction sends that transition to a new absorbing success state and computes its Expected hitting time.
For the two-state chain , , with , the waiting time for has means and . Starting at one, the hitting time of state one is zero, whereas the transition-completion time must first wait for and then . This example makes the difference between an edge event and a state event explicit.
Two-state excursion visit law Created 2026-09-24 Updated 2026-11-05
Let belong to an irreducible recurrent Discrete-time Markov chain. Write and . Define
The positive return convention is essential: itself is zero when starting at . Let count visits to before the first positive return to , including repeated holding visits at . Then
With Probability the excursion reaches . From each visit there, the Strong Markov property gives Probability of hitting before the next return to . recurrence ensures there is no third possibility of escaping both states forever. Thus, conditional on reaching , the visit count has a positive-support geometric distribution with mean .
For a two-state chain which chooses either state independently with Probability at every step, . Half of the return cycles from zero never visit one; the rest visit it a geometrically distributed number of times of mean two. The unconditional expected visit count is therefore one.
Refill the stock to a fixed positive size immediately whenever a step empties it. The resulting Markov chain has state space and transitions for and . It is irreducible and aperiodic, since every state reaches , reaches every state, and . Its unique stationary distribution is and for . Normalization is the harmonic number identity. For , stationarity follows from ; the same telescoping sum starting at gives stationarity at . The finite-state convergence theorem therefore makes this stationary distribution the limiting distribution from every initial state.
An irreducible Markov chain on a finite or countably infinite state space has at most one stationary distribution. On a countably infinite state space it exists exactly when the chain is positive recurrent. In that case, for every state ,
where is the first return time. The expression determines every component uniquely. The formula follows by dividing the one visit to in each return cycle by the mean recurrence time; the strong law of large numbers for successive cycles makes this occupation fraction precise.
Every irreducible finite Markov chain is positive recurrent and has exactly one stationary distribution. On an infinite state space, an irreducible null recurrent Markov chain or transient Markov chain has none. For example, the simple symmetric random walk on the integers is null recurrent. Aperiodicity affects convergence of the one-time laws, but is unnecessary for existence or uniqueness of a stationary distribution.