Birth-death recurrence classification Created 2026-09-24 Updated 2026-11-05
Consider a birth-death process on with constant birth rate , constant death rate at every positive state, and no deaths from zero. The reflecting boundary is essential: the result changes if zero is an absorbing state. Detailed balance would giveWhen , these weights normalize to a Geometric distribution, so the irreducible chain is positive recurrent. When , the embedded jump chain is the symmetric reflected random walk: it returns to zero almost surely, but its constant equilibrium weights have infinite sum, so the chain is null recurrent. When , its upward-biased embedded random walk has probability of hitting zero from state one, making the chain transient. The holding rates are bounded, so nonexplosion creates no additional qualification here.
Dirichlet energy space with zero boundary at infinity Created 2026-09-24 Updated 2026-11-05
For a transient weighted graph, is the completion of the finitely supported functions in the norm . Its elements have finite Dirichlet energy and vanish at infinity in the energy sense.
Explosion of an upward-biased walk with geometrically increasing rates Created 2026-09-24 Updated 2026-11-05
Suppose a transient nearest-neighbour jump chain on has uniformly bounded expected visits to each state and the holding rate at grows geometrically. Thenso the continuous-time chain explodes almost surely.
Geometric-arrival single-service queue Created 2026-10-14 Updated 2026-11-05
Let the independent arrivals satisfy , with and . During a busy period, each served customer produces further customers, so the total number served has the total-progeny law of a Galton-Watson process with mean . Its extinction probability solves and is the smaller of and . The queue is positive recurrent for , null recurrent for , and transient for . In the positive recurrent case its stationary distribution is .
Heavy-traffic limit of a queue Created 2026-10-14 Updated 2026-11-05
For fixed independent and identically distributed arrivals of mean and positive finite variance , put service at . Rescaling net input by space and time gives Brownian motion with drift by the functional central limit theorem. The continuous Skorokhod reflection map on the half-line gives the displayed transient limit, with a correspondingly scaled initial workload. A stationary approximation additionally requires justification of limit interchange. When valid, the stationary law of negatively drifted reflected Brownian motion predicts .
Hitting probabilities for a velocity-switching Markov ladder Created 2026-10-14 Updated 2026-11-05
Consider states for integer levels and strictly positive rates . From the chain moves to at rate and to at rate . From , , it moves to at rate or to at rate . At the reflecting boundary there is only the switch to ; no negative level is allowed. All rates are bounded, so the chain is nonexplosive.
Let be the probability of hitting , with . First-step equations arewith the first equation also valid at . Put . If , then andSubstitution verifies both equations and the boundary. Eliminating the variables gives a second-order recurrence with modes . The displayed decaying solution is the minimal nonnegative harmonic solution and hence the Hitting probability.
When , the growing mode is excluded by boundedness; at equality the second mode is linear and is likewise excluded. The only bounded solution with the boundary value is one, proving almost-sure hitting. Since the specified reflected chain is irreducible, these cases classify it as recurrent or transient respectively. Specifying the boundary rule is essential for that recurrence statement; making absorbing would be a different chain.
Open communicating class Created 2026-10-06 Updated 2026-11-05
A communicating class of a Markov chain is open when a state in the class has positive Probability of transitioning to a state outside it. Once a path leaves, it cannot return to the class: a return path would make the outside state communicate with the class. Every open class consists of transient states. Conversely, a finite transient communicating class is open; an infinite closed communicating class can be transient.
Start the Markov chain at a state and define the strictly positive first return time . A recurrent state satisfies ; a transient state has this probability strictly below one. Equivalently a recurrent state is visited infinitely often almost surely when started there, whereas a transient state receives finitely many visits almost surely. The recurrence criterion by return probabilities gives the equivalent testStates in a communicating class share this classification. In an irreducible Markov chain all states communicate, so the chain is called a recurrent Markov chain if all its states are recurrent and a transient Markov chain if all are transient.
Preservation under enlargement of transition support Created 2026-09-24 Updated 2026-11-05
Suppose two finite Markov chains on the same state space have transition matrices satisfyingEvery positive-probability path and return cycle for is also possible for . Thus irreducibility passes from to . For a state aperiodic under , its positive return lengths under contain those under , so their greatest common divisor is still one. Hence aperiodicity also passes to .
Individual-state recurrence need not be preserved when the original chain is reducible. For example, take on andState zero is recurrent under but transient under : once the chain leaves it, it never returns. The added transition opens an escape from its former closed class. If is finite and irreducible, however, then is also finite and irreducible, so every state remains positive recurrent. Mean recurrence times can still change because the Stationary distribution depends on transition probabilities, not merely on their support.
Recurrence characterized by constant nonnegative superharmonic functions Created 2026-10-14 Updated 2026-11-05
An irreducible Markov chain is recurrent exactly when every nonnegative superharmonic function for a Markov chain is constant. In the recurrent case, absorb the chain at . The inequality and almost-sure hitting give ; exchange . In the transient case, the hitting probability of a state is one there and satisfies the superharmonic inequality, but its one-step average at is the strictly smaller return Probability. It is a nonconstant example.
Reflected chain with geometric downward jumps Created 2026-10-14 Updated 2026-11-05
Fix and put . For independent geometric random variables with for , the Reflected chain with geometric downward jumps obeys . Its transition probabilities are for and .
The Probability of hitting zero from is one when and when . The first-step boundary equation at is essential for the extra exponent. Before reflection the mean increment is : downward drift corresponds to .
The chain is positive recurrent for , with Stationary distribution for . The ratio ensures these weights sum to one. It is null recurrent at and transient for . The parameter restriction matters: does not define the displayed geometric distribution, and produces deterministic upward motion rather than an irreducible chain.
Take independent copies of the simple random walk on the integer line, each stepping left or right with equal probability at every integer time. Their product Markov chain moves all coordinates at once; each of its diagonal increments has probability . At odd times a return to the origin is impossible. At time , independence gives the displayed return probability, whose asymptotic equivalence follows from Stirling's approximation. The recurrence criterion by return probabilities and the p-series therefore imply recurrence for and a transient Markov chain for .
For the chain from the origin visits only the sublattice whose coordinates all have the same parity. This is one communicating class; restricting to it gives an irreducible Markov chain. The period of a state in a Markov chain is two here. The conclusion applies to that class, without incorrectly claiming Markov chain irreducibility on all of . This walk moves diagonally; the usual axis-step simple random walk has different transition probabilities, even though the same dimensional recurrence threshold holds.
Tournament won by two consecutive victories Created 2026-10-06 Updated 2026-11-05
With three evenly matched players, the previous winner plays the previous spectator. A pair of successive distinct winners moves to or with equal probabilities, where is the remaining player. The transient states form two directed 3-cycles and each step has absorption Probability one half. Starting with a game between two specified players, their overall winning probabilities are each , while the original spectator wins with Probability .
Transient communicating class Created 2026-09-24 Updated 2026-11-05
In a finite chain, a Communicating class that is not closed is transient: after it is left, it need not be visited again and is eventually left almost surely.
Transient Markov chain Created 2026-10-31 Updated 2026-11-05
An irreducible Markov chain is transient when its states are transient states. Starting at any one state, there is a positive Probability of never returning, and the number of visits to any fixed state is finite almost surely.
Transient state Created 2026-09-24 Updated 2026-11-05
An irreducible Markov chain on a finite or countably infinite state space has at most one stationary distribution. On a countably infinite state space it exists exactly when the chain is positive recurrent. In that case, for every state ,where is the first return time. The expression determines every component uniquely. The formula follows by dividing the one visit to in each return cycle by the mean recurrence time; the strong law of large numbers for successive cycles makes this occupation fraction precise.
Every irreducible finite Markov chain is positive recurrent and has exactly one stationary distribution. On an infinite state space, an irreducible null recurrent Markov chain or transient Markov chain has none. For example, the simple symmetric random walk on the integers is null recurrent. Aperiodicity affects convergence of the one-time laws, but is unnecessary for existence or uniqueness of a stationary distribution.