Starting from a state , let be the first return time, with value infinity if no return occurs. The state is recurrent if and transient if this probability is less than 1. A recurrent state is positive recurrent if , and null recurrent if that expectation is infinite.
If and communicate, choose with and . The Chapman-Kolmogorov equations give, for every ,
If is recurrent, the supplied criterion makes infinite. Summing this inequality shows that is also infinite, so is recurrent. The same argument with the states exchanged gives the converse. Hence recurrence and transience are class properties: states in one communicating class cannot have different answers to that dichotomy.
A state is recurrent if the probability of returning to at a strictly positive time, starting at , equals one. It is transient if that probability is less than one. If and communicate, choose with and . Chapman-Kolmogorov gives
If is recurrent, summing over makes the right-hand side diverge, so the allowed recurrence criterion proves recurrent. Interchanging proves recurrence is shared throughout a communicating class, and therefore transience is shared too.
For the simple random walk, an odd number of steps cannot return to its starting point, and
For the return series diverges like , proving recurrence. For , , its tail is bounded by a convergent geometric series since , proving transience. At or the walk is deterministic and never returns. Translation invariance transfers the conclusion to every state. Thus recurrence holds exactly when .
A state is recurrent if the probability of returning to at a strictly positive time, starting at , equals one. It is transient if that probability is less than one. If and communicate, choose with and . Chapman-Kolmogorov gives
If is recurrent, summing over makes the right-hand side diverge, so the allowed recurrence criterion proves recurrent. Interchanging proves recurrence is shared throughout a communicating class, and therefore transience is shared too.
For the simple random walk, an odd number of steps cannot return to its starting point, and
For the return series diverges like , proving recurrence. For , , its tail is bounded by a convergent geometric series since , proving transience. At or the walk is deterministic and never returns. Translation invariance transfers the conclusion to every state. Thus recurrence holds exactly when .
Start the Markov chain at a state and define the strictly positive first return time . A recurrent state satisfies ; a transient state has this probability strictly below one. Equivalently a recurrent state is visited infinitely often almost surely when started there, whereas a transient state receives finitely many visits almost surely. The recurrence criterion by return probabilities gives the equivalent test
States in a communicating class share this classification. In an irreducible Markov chain all states communicate, so the chain is called a recurrent Markov chain if all its states are recurrent and a transient Markov chain if all are transient.
This is the synchronous product of simple symmetric random walks, rather than a walk that changes only one coordinate at a time. Independence of the three coordinate processes gives
Odd-time return probabilities vanish. The p-series with exponent converges, so the sum of all return probabilities is finite. By the recurrence criterion by return probabilities, the origin is a transient state: the three-dimensional process is transient. From the origin its communicating class consists of integer triples whose coordinates have the same parity; translation gives the same classification for every other class.
A transition matrix defines an irreducible Markov chain if every state is accessible from every other state:
Equivalently, all states belong to one communicating class. When , a positive number of steps is necessarily used.
Define
An interior jump cannot go below or above , and every exterior reset lands in . Thus is closed.
To prove that it is one communicating class, start anywhere in . From or , a reset can reach with positive probability. From any state of , repeated steps reach some state of , after staying in up to the last step. A reset from this state can reach every point of the consecutive block
Every integer has the form for some and an integer . Repeated steps from reach , with all departure states in until a possible last step into . This gives access to all of . Finally, each can be reached from . Every path described has strictly positive probability. Therefore every state in reaches every other state in .
Each state outside can only stay put or reset into , and no transition from returns to it. Different states outside cannot reach one another. Hence the complete list of communicating classes is
The finite class is the unique closed communicating class. The exterior singleton classes are transient communicating classes, since their self-transition probabilities are strictly less than one. Notice that this reachability proof does not require to be coprime.
Let and put , . Then are coprime integers, and . The six requested revisions are as follows.
(b)(i): communicating classes. Every increment is divisible by , so the residue class modulo is preserved. Conversely, if is divisible by , Bézout's identity applied to gives a path just as in (b)(i). Thus the original Markov chain is reducible on , with exactly the closed communicating classes
The specified walk starting at zero stays in .
(b)(ii): period of a state in a Markov chain. The return equation reduces to . Hence , . For every state, the period of a state in a Markov chain is therefore
This is still at least two, so no state is aperiodic.
(b)(iii): stationary distribution. The conclusion is unchanged: there is no stationary distribution on . The maximum-mass proof in (b)(iii) applies verbatim, without using coprimality. Equivalently, each closed communicating class, after translating and dividing its state labels by , is the same centered random walk with step sizes and has no stationary distribution.
(b)(iv): occupation of zero. On , the rescaled walk has exactly the transition probabilities of the coprime walk with parameters . The return-probability argument in (b)(iv) therefore makes this class null recurrent. The renewal process proof of the occupation limit applies unchanged:
Starting in another residue class would make every visit to zero impossible, also giving occupation proportion zero.
(c)(i): modified communicating classes. These are unchanged: is one finite closed communicating class, and all other states are transient singleton classes. The original proof used consecutive reset destinations and never used coprimality. Resetting to the consecutive integers connects different residue classes, so it would be incorrect to split according to residue classes modulo .
(c)(ii): limiting distribution. The column-sum calculation, the irreducible Markov chain property on a finite class, positive self-transition probabilities, and geometric entrance argument also required no coprimality. Thus for every initial state the same convergence in distribution holds:
In particular, the denominator is unchanged; it is not divided by .
The Markov property says that, conditional on the present state, the next state is independent of the earlier history:
Two states communicate if each is reachable from the other with positive probability in finitely many steps. The equivalence classes are communicating classes; a class is closed if no positive-probability transition leaves it.
The displayed probabilities require ; negative values allowed by the printed one-sided inequality are not transition probabilities. For , every state is an absorbing singleton, so all singleton classes are closed. For , neighboring states communicate in both directions, so there is one closed irreducible class and no open classes.
For , the detailed balance relations give . Normalization yields
For , positive holding probabilities make the finite irreducible chain aperiodic, so its transition probabilities converge to from every initial state. At , every step changes parity and the period is two; convergence from a fixed initial state fails. A stationary initial mixture still has distribution at every time, but this does not give convergence of the transition probabilities asked for.
A state is recurrent if, starting there, the probability of returning at some strictly positive time is one. It is transient if that return probability is strictly less than one. Suppose communicate. Choose with and . Chapman–Kolmogorov gives
If is recurrent, summing over and using the supplied return-series criterion makes the return series for diverge, so is recurrent. Interchanging the states proves equivalence; therefore transience is also constant on a communicating class.
For the nearest-neighbor walk, return in an odd number of steps is impossible. A return in steps requires exactly steps each way, so
At , the return series diverges like . If and , then , so it converges by exponential decay. At or one the walk never returns. Hence .
Periodic Markov chain Created 2026-09-24 Updated 2026-11-05
For a state of a Markov chain, its period is the greatest common divisor of the positive return lengths,
A state is periodic when . All states in an irreducible class have the same period: paths connecting two states transfer return cycles between them. A reducible chain can have classes of different periods, so a single period for the whole chain requires qualification.
The deterministic two-state chain has period two: its distributions alternate forever despite the existence of the uniform Stationary distribution. More generally, an irreducible nearest-neighbour walk on a bipartite graph alternates its two vertex classes. A positive holding probability at any state of an irreducible class supplies a one-step return and makes the entire class aperiodic. Aperiodicity therefore concerns possible return lengths rather than merely the existence of an invariant law.
Positive recurrent Markov chain Created 2026-10-06 Updated 2026-11-05
An irreducible Markov chain is positive recurrent when its mean return times are finite. For a nonexplosive Markov chain on a countable state space, this is equivalent to the existence of a stationary distribution on its irreducible class. For continuous time, a return cycle includes departure and return, rather than counting immediate residence as a return.
Recurrent Markov chain Created 2026-09-24 Updated 2026-11-05
A Recurrent Markov chain returns to its states rather than eventually leaving them forever. A state is recurrent if its positive first return time satisfies . A chain is recurrent when every state is recurrent. Within an irreducible class, recurrence is shared by every state, so testing one state suffices.
The Strong Markov property at successive return times shows that a Recurrent state is visited infinitely often almost surely when starting there. If its return Probability is , the number of visits including time zero has a geometric distribution on the positive integers, with mean . Consequently the Green-function criterion is
Tonelli's theorem identifies this sum with the expected number of visits. recurrence does not require a finite mean return time: null recurrence and positive recurrence distinguish infinite from finite mean recurrence time. simple symmetric random walk on the integers is null recurrent, whereas every finite irreducible chain is positive recurrent.
Return-free state in a Markov chain Created 2026-10-14 Updated 2026-11-05
A state can have no positive-probability return at any positive time, for example when its transition-matrix column is zero. Its return-time set is empty. The convention assigns period zero; conventions defining period only for a nonempty return set instead leave it undefined. Such a state must not be called aperiodic merely because its Communicating class is a singleton.
Take independent copies of the simple random walk on the integer line, each stepping left or right with equal probability at every integer time. Their product Markov chain moves all coordinates at once; each of its diagonal increments has probability . At odd times a return to the origin is impossible. At time , independence gives the displayed return probability, whose asymptotic equivalence follows from Stirling's approximation. The recurrence criterion by return probabilities and the p-series therefore imply recurrence for and a transient Markov chain for .
For the chain from the origin visits only the sublattice whose coordinates all have the same parity. This is one communicating class; restricting to it gives an irreducible Markov chain. The period of a state in a Markov chain is two here. The conclusion applies to that class, without incorrectly claiming Markov chain irreducibility on all of . This walk moves diagonally; the usual axis-step simple random walk has different transition probabilities, even though the same dimensional recurrence threshold holds.
Transient communicating class Created 2026-09-24 Updated 2026-11-05
In a finite chain, a Communicating class that is not closed is transient: after it is left, it need not be visited again and is eventually left almost surely.