A countable nonexplosive continuous-time Markov chain which is an irreducible Markov chain and has an invariant distribution is positive recurrent; its transition probabilities tend to from every initial Markov chain state.
To see why continuous time removes periodicity, fix . Every finite sequence of Markov chain states connected by positive transition intensities has positive probability of finishing before , followed by no further transition until . Consequently the transition matrix has positive entries between every pair of communicating Markov chain states, including its diagonal. It is an irreducible Markov chain and an aperiodic Markov chain with stationary distribution . The countable-state Markov chain convergence theorem gives convergence at times , also in total variation distance. For , , applying contracts total variation distance and preserves , giving the full continuous-time limit.
Past exam of the mathematics course of the University of Cambridge 2016 ib Paper 2 20H b iii Solution Created 2026-09-24 Updated 2026-10-06
The only closed communicating class is ; the class is open and transient. Its one-step probability of staying inside is at most , so if is the entrance time into , then for an initial state in . Hence entrance occurs almost surely in finite time.
On it is an irreducible aperiodic Markov chain, because both states have positive self-loop probabilities. The finite irreducible aperiodic countable-state Markov chain convergence theorem says that its transition probabilities converge to its unique stationary distribution. The balance equation with gives and .
Past exam of the mathematics course of the University of Cambridge 2017 ib Paper 2 20H c Solution Created 2026-09-24 Updated 2026-10-05
Every reachable state reaches zero deterministically in steps. For , choose a supported lifetime ; zero can jump to and then count down to . Thus the chain is an irreducible Markov chain on . Its positive-probability first return lengths are exactly the support of ; all other return lengths are sums of such lengths. Their greatest common divisor is therefore one by assumption, so the chain is an aperiodic Markov chain.
The equilibrium residual-life distribution isIt is a probability distribution because the tail-sum identity gives . With zero masses outside ,so it is a stationary distribution. This also displays why finite mean is needed for normalization.
Use the countable-state Markov chain convergence theorem: an irreducible, aperiodic, positive recurrent discrete-time chain on a countable state space has a unique stationary distribution and for every pair of states. All hypotheses were verified, including positive recurrence in part (b). Since and ,The finite-state convergence theorem alone would not suffice when the lifetimes are unbounded. The gcd hypothesis is also necessary for this ordinary limit: deterministic lifetime 2, for example, produces alternating renewal probabilities rather than convergence.
For each adjacent pair, the detailed balance equations follows directly:For nonadjacent distinct states both fluxes vanish, and diagonal fluxes agree trivially. Since , this is a genuine stationary distribution, rather than merely an invariant measure of infinite mass.
Every upward and downward transition probability between neighbouring states is positive for . Any two states can therefore be joined by a finite path of positive probability: the Markov chain is irreducible. Moreover,State zero has period one; Markov chain irreducibility makes every state aperiodic. Finally, the standard countable-state recurrence theorem states that an irreducible Markov chain admitting a stationary distribution of total mass one is positive recurrent, with that stationary distribution unique. The countable-state Markov chain convergence theorem then states that Markov chain irreducibility, positive recurrence and aperiodicity imply for all states . The hypotheses just verified hold on the infinite state space , not merely on a numerical truncation.[ref]
Markov Chains by James R. Norris. Sections 1.7–1.8, invariant distributions and convergence to equilibrium
Consequently, the unique stationary distribution and the limiting distribution are both , from every initial state. The countable-state Markov chain ergodic theorem also gives almost surely whenever .[ref]
The update defines a time-homogeneous Markov process: given its present value, the next value depends only on independent fresh input. Part (b)(i) identifies an invariant law: initializing with the rate-one exponential distribution preserves that distribution at every time. Part (b)(iii) shows convergence of the one-time laws to this invariant law from every deterministic initial value.
For a countable state space, the analogous countable-state Markov chain convergence theorem says that an irreducible Markov chain that is aperiodic and positive recurrent converges to its unique stationary distribution from every initial state. Here the state space is continuous, so that countable-state theorem does not directly supply the proof; the explicit coupling does. The two conclusions have the same equilibrium interpretation.
The restriction is a finite Markov chain that is an irreducible Markov chain. It is aperiodic, because its upper states have self-transition probability , and the period is constant on a communicating class. We now find its stationary distribution.
Every column of sums to one. Here is the full incoming-mass check, with each total measured in units of . For , the incoming contributions are from and from the self-transition. For , they are from and from the self-transition.
For , the upward predecessor lies in exactly when , contributing units. When , that predecessor is replaced by resets from all states of , again contributing units. Independently, the downward predecessor contributes units when ; when , resets from the states of contribute those units instead. Thus every column total is units.
Therefore is a doubly stochastic matrix, and the uniform distribution on is stationary:The countable-state Markov chain convergence theorem, applied to the finite closed class , gives for every . Its hypotheses hold: is finite and irreducible, hence positive recurrent, and it is aperiodic.
If , the hitting time of has . If , the corresponding tail is . Thus almost surely from every exterior initial state. Conditioning on and , use convergence within after entrance and the bounded convergence theorem; equivalently, first restrict to and then let . This proves, for every ,The limiting distribution is independent of .
For the countable-state Markov chain ergodic theorem, let the Markov chain be irreducible and positive recurrent, with its unique stationary distribution . From any initial state, for every real satisfying ,In particular, almost surely. This time-average conclusion requires no aperiodicity.
The countable-state Markov chain convergence theorem adds the hypothesis of an aperiodic Markov chain and concludesIt also follows that for every initial probability distribution . For an irreducible recurrent chain, the occupation-frequency theorem can also be stated as a limit , interpreted as zero for a null recurrent state; in the positive recurrent case this equals .
Every step changes the population by one, so has the parity of when . Hence . A return in two steps has positive probability from every state, and all return times are even, so the period of a state in a Markov chain is exactly two.
Each transition exchanges the even and odd classes. Since is an invariant distribution, their masses are equal: . The chain sampled at times , restricted to the even states, is irreducible and positive recurrent, and it is aperiodic because its diagonal entries are positive. Its stationary distribution is on even . The countable-state Markov chain convergence theorem applied to this sampled chain yieldsSince , the full sequence has no limit. Its time average nevertheless tends to by the countable-state Markov chain ergodic theorem.
Poisson Metropolis chain 2026-10-25
A Poisson Metropolis chain samples a Poisson distribution by proposing an upward or downward unit step with equal probability and reflecting a downward proposal at zero into a self-proposal. Its upward and downward transition probabilities are the displayed , with , and the diagonal entry is . For , adjacent stationary fluxes equal . Hence detailed balance holds. Positive adjacent probabilities imply an irreducible Markov chain; implies aperiodicity. The normalized invariant Poisson distribution implies positive recurrence, so the countable-state Markov chain convergence theorem applies. The boundary proposal probability is essential: forcing every proposal at zero to go upward would require a different acceptance ratio.
Stationary distribution Created 2026-09-24 Updated 2026-10-14
A stationary distribution is a probability distribution preserved by a Markov chain's evolution. For a Discrete-time Markov chain in the row convention,Thus is a normalized nonnegative left eigenvector of the transition matrix for eigenvalue one. Starting with distributed as makes every have that same probability distribution. For a time-homogeneous Markov chain, the path probabilities are also unchanged by shifting all observation times together: the process has strict stationarity. This does not mean its individual paths are constant. See Hao Wu's MIT lecture 2, stationary distributions.
For the sunny/rainy transition matrix, write . The sunny component of gives , so andMoreover,These equal stationary flows verify detailed balance and show that the Markov chain is reversible. The diagonal detailed balance equations hold automatically. Only of the stationary days are rainy, but of the days following a rainy day are rainy: stationarity permits dependence.
A finite Markov chain that is an irreducible Markov chain has a unique stationary distribution. If it is also an aperiodic Markov chain, for every initial probability distribution. The sunny/rainy example satisfies both conditions: both directions are possible and each state has a positive holding probability of a Markov chain. See Hao Wu's MIT lecture 4, convergence theorem. For an infinite state space, the additional positive recurrence hypothesis in the countable-state Markov chain convergence theorem matters.
Existence of a stationary distribution alone does not give this convergence. Consider the deterministic two-cycleHere , but starting in the first state gives alternating probability distributions . Its period of a state in a Markov chain is two. The running fraction of visits to either state nevertheless tends to , illustrating the Markov chain ergodic theorem. Starting with instead keeps the one-time probability distributions stationary even though every path alternates.
