Stationary distribution ($\pi P=\pi$)

A stationary distribution is a probability distribution $\pi$ preserved by a Markov chain's evolution. For a Discrete-time Markov chain in the row convention,Thus $\pi$ is a normalized nonnegative left eigenvector of the transition matrix for eigenvalue one. Starting with $X_0$ distributed as $\pi$ makes every $X_n$ have that same probability distribution. For a time-homogeneous Markov chain, the path probabilities are also unchanged by shifting all observation times together: the process has strict stationarity. This does not mean its individual paths are constant. See Hao Wu's MIT lecture 2, stationary distributions.

For the sunny/rainy transition matrix, write $\pi =(a,1-a)$. The sunny component of $\pi P=\pi$ gives $a=0.8a+0.4(1-a)$, so $0.6a=0.4$ andMoreover,These equal stationary flows verify detailed balance and show that the Markov chain is reversible. The diagonal detailed balance equations hold automatically. Only $1/3$ of the stationary days are rainy, but $0.6$ of the days following a rainy day are rainy: stationarity permits dependence.

A finite Markov chain that is an irreducible Markov chain has a unique stationary distribution. If it is also an aperiodic Markov chain, ${\mu }_0{P}^n\to \pi$ for every initial probability distribution. The sunny/rainy example satisfies both conditions: both directions are possible and each state has a positive holding probability of a Markov chain. See Hao Wu's MIT lecture 4, convergence theorem. For an infinite state space, the additional positive recurrence hypothesis in the countable-state Markov chain convergence theorem matters.

Existence of a stationary distribution alone does not give this convergence. Consider the deterministic two-cycleHere $\pi C=\pi$, but starting in the first state gives alternating probability distributions $(1,0),(0,1),(1,0),\dots$. Its period of a state in a Markov chain is two. The running fraction of visits to either state nevertheless tends to $1/2$, illustrating the Markov chain ergodic theorem. Starting with $\pi$ instead keeps the one-time probability distributions stationary even though every path alternates.

If a distinguished book is selected with probability $p$ and each other book with probability $q=(1-p)/(N-1)$, swapping the selected book with its left neighbor makes the distinguished position a birth-death chain with left rate $p$ and right rate $q$. Stationary flow balance gives successive probabilities in ratio $q/p$, yielding a truncated geometric distribution; the uniform case occurs when all books are equally likely.

On states $0,\dots ,m$, take an interior step of $+1$ or $-1$ with probability $p$ each and hold with probability $1-2p$. At an endpoint take the inward step with probability $2p$ and hold with probability $1-2p$. For $m\ge 1$ and $0<p\le 1/2$, the chain is irreducible and its stationary distribution has endpoint weights $1/(2m)$ and interior weights $1/m$.

Detailed balance proves this: the doubled inward endpoint transition requires its stationary mass to be half that of an adjacent interior point, and all interior points have equal mass. The total unnormalized weight is $1+2(m-1)+1=2m$. For $m=1$ there are just two equal endpoint masses.

When $p<1/2$, positive self-loops make the chain aperiodic, so its transition probabilities converge to this law. At $p=1/2$ every move changes parity and two-step returns are possible, giving period two and no such convergence from a fixed starting state. At $p=0$ every state is absorbing and uniqueness of the stationary law is lost.

An irreducible Markov chain on a finite or countably infinite state space has at most one stationary distribution. On a countably infinite state space it exists exactly when the chain is positive recurrent. In that case, for every state $i$,where $T_i^{+}$ is the first return time. The expression determines every component uniquely. The formula follows by dividing the one visit to $i$ in each return cycle by the mean recurrence time; the strong law of large numbers for successive cycles makes this occupation fraction precise.

Every irreducible finite Markov chain is positive recurrent and has exactly one stationary distribution. On an infinite state space, an irreducible null recurrent Markov chain or transient Markov chain has none. For example, the simple symmetric random walk on the integers is null recurrent. Aperiodicity affects convergence of the one-time laws, but is unnecessary for existence or uniqueness of a stationary distribution.

Every finite Markov chain has a stationary distribution, without requiring Markov chain irreducibility or aperiodicity. For a row-stochastic matrix $P$, put $B=P^T$ and choose a column probability vector $q$. Every $B^{j}q$ lies in the probability simplex. Their arithmetic averages $x_N=N^{-1}{\sum }_{j=0}^{N-1}{B}^{j}q$ therefore lie in this compact set, while telescoping givesA convergent subsequence has a limit $x$ in the probability simplex. Continuity of matrix multiplication gives $Bx=x$, so $\pi =x^T$ is a stationary distribution. The individual iterates need not converge: a deterministic two-state swap alternates its point masses forever.

For a discrete-time Markov chain with transition matrix $P$, these equations, together with ${\pi }_j\ge 0$ and ${\sum }_j{\pi }_j=1$, say that $\pi$ is a stationary distribution. Incoming probability mass equals the current mass at each state. They are global balance equations and do not require detailed balance.

Suppose each coordinate has stationary distribution $\pi$ for the same Markov kernel. Updating one coordinate using independent new randomness while leaving the others fixed preserves the product distribution ${\pi }^{\otimes m}$. For a two-state update with $P(1\to 1)=a$ and $P(0\to 1)=b$, its stationary occupied probability is $p=b/(1-a+b)$. Independent occupied-state indicators then have total mean $mp$ and variance $mp(1-p)$. The conclusion requires the update selection and transition randomness not to introduce dependence on the other coordinates' states.