Reflected lazy walk with doubled endpoint transitions Created 2026-10-31 Updated 2026-11-05
On the finite path , take interior neighbor probabilities , endpoint inward Probability , and holding Probability , with . Detailed balance gives the displayed invariant distribution: endpoint weights are half the interior weights. For the chain is irreducible and aperiodic, so transition probabilities converge to it. At parity alternates and the period is two.
Return-free state in a Markov chain Created 2026-10-14 Updated 2026-11-05
A state can have no positive-probability return at any positive time, for example when its transition-matrix column is zero. Its return-time set is empty. The convention assigns period zero; conventions defining period only for a nonempty return set instead leave it undefined. Such a state must not be called aperiodic merely because its Communicating class is a singleton.
Stationary cycle occupation formula Created 2026-09-24 Updated 2026-11-05
For an irreducible positive recurrent Discrete-time Markov chain with Stationary distribution , fix a state and its positive first return . The Stationary cycle occupation formula isThe cycle includes the initial visit to and excludes the returning visit. Thus for the left side is exactly one, consistent with the ratio.
Successive return cycles are independent with the same law by the Strong Markov property. The renewal reward ratio gives the long-run visit fraction as expected cycle occupation divided by expected cycle length. Kac's lemma supplies , so the ergodic visit fraction gives the displayed identity. positive recurrence ensures the cycle length and all its occupation rewards are integrable.
For distinct , the Two-state excursion visit law gives the same expected occupation as , where and . ConsequentlyThis identity does not require reversibility: it follows from return-cycle occupation. It also does not require aperiodicity, since time averages still converge in a periodic positive recurrent chain.
Stationary distribution Created 2026-09-24 Updated 2026-10-14
A stationary distribution is a probability distribution preserved by a Markov chain's evolution. For a Discrete-time Markov chain in the row convention,Thus is a normalized nonnegative left eigenvector of the transition matrix for eigenvalue one. Starting with distributed as makes every have that same probability distribution. For a time-homogeneous Markov chain, the path probabilities are also unchanged by shifting all observation times together: the process has strict stationarity. This does not mean its individual paths are constant. See Hao Wu's MIT lecture 2, stationary distributions.
For the sunny/rainy transition matrix, write . The sunny component of gives , so andMoreover,These equal stationary flows verify detailed balance and show that the Markov chain is reversible. The diagonal detailed balance equations hold automatically. Only of the stationary days are rainy, but of the days following a rainy day are rainy: stationarity permits dependence.
A finite Markov chain that is an irreducible Markov chain has a unique stationary distribution. If it is also an aperiodic Markov chain, for every initial probability distribution. The sunny/rainy example satisfies both conditions: both directions are possible and each state has a positive holding probability of a Markov chain. See Hao Wu's MIT lecture 4, convergence theorem. For an infinite state space, the additional positive recurrence hypothesis in the countable-state Markov chain convergence theorem matters.
Existence of a stationary distribution alone does not give this convergence. Consider the deterministic two-cycleHere , but starting in the first state gives alternating probability distributions . Its period of a state in a Markov chain is two. The running fraction of visits to either state nevertheless tends to , illustrating the Markov chain ergodic theorem. Starting with instead keeps the one-time probability distributions stationary even though every path alternates.
On states , take an interior step of or with probability each and hold with probability . At an endpoint take the inward step with probability and hold with probability . For and , the chain is irreducible and its stationary distribution has endpoint weights and interior weights .
Detailed balance proves this: the doubled inward endpoint transition requires its stationary mass to be half that of an adjacent interior point, and all interior points have equal mass. The total unnormalized weight is . For there are just two equal endpoint masses.
When , positive self-loops make the chain aperiodic, so its transition probabilities converge to this law. At every move changes parity and two-step returns are possible, giving period two and no such convergence from a fixed starting state. At every state is absorbing and uniqueness of the stationary law is lost.
Three-level no claims discount equilibrium Created 2026-10-14 Updated 2026-11-05
For a three-level birth-death chain whose down-or-stay claim probabilities are , with each in , detailed balance is and . Normalizing gives the displayed stationary distribution. The finite chain is irreducible and has endpoint holding probabilities, so it is aperiodic and its level proportions converge to this distribution from any initial level.
Uniform depletion and refill Markov chain 2026-10-31
Refill the stock to a fixed positive size immediately whenever a step empties it. The resulting Markov chain has state space and transitions for and . It is irreducible and aperiodic, since every state reaches , reaches every state, and . Its unique stationary distribution is and for . Normalization is the harmonic number identity. For , stationarity follows from ; the same telescoping sum starting at gives stationarity at . The finite-state convergence theorem therefore makes this stationary distribution the limiting distribution from every initial state.
