For a Markov chain started at state , let denote the first positive return time, using time for the starting state. A recurrent state has ; a transient state has return probability strictly less than . An absorbing state satisfies , so the chain remains there forever after arrival and it is necessarily a recurrent state.
State is an absorbing state and a recurrent state. From every positive state, the stock decreases strictly at the next step and can never increase, so a return to that positive state is impossible. Thus all positive states are transient, with return probability zero. Starting from , absorption occurs within at most sticker-taking children.
An absorbing state has ; once reached, the chain never leaves it. For a nonabsorbing initial state, condition on its first transition and use the Markov property:
Here and . The other equations are
Solving gives
For absorption into either state one or four, let be the expected number of steps, with . first-step analysis gives
Hence and
The expectation is finite because each transient two-step cycle has probability at most of remaining transient; the survival tail is geometrically bounded. This also justifies solving the expectation equations without assuming finiteness in advance.
A state is a recurrent state if the probability of a return at some positive time, starting there, is one. It is a transient state if that return probability is less than one. It is an absorbing state if its one-step probability of remaining there is one. An absorbing state is recurrent, since every subsequent time is a return.
The absorbing states give . Multiplying the difference equation in (i) by gives
The second finite difference of is , so a particular solution is ; the homogeneous solutions are . Imposing the two boundary conditions gives . Therefore
Uniqueness follows because the difference of any two solutions has zero second finite difference, hence is affine function, and vanishes at both endpoints. This is the mean absorption time of a lazy symmetric random walk with holding probability .
Let . By the law of total probability,
There is no return contribution from the absorbing state . Since can be entered only from , for . Substitution yields
No independence between successive positions is assumed; the conditional transitions give the recurrence.
This continuous-time multi-state model permits progression , direct death , and death after progression , with no recovery and death an absorbing state. For constant positive rates , write . Integrating over the time of the progression arrow gives
Use when . The other transition probabilities are , , , and . Unlike a purely sequential irreversible three-state disease model, the direct-death rate contributes to the first-state exit rate. A mixed panel and exact-death likelihood accounts for panel visits and exact death times with different kinds of factors.
A sequential continuous-time Markov chain has transitions with rates and an absorbing state . Its probability of ever leaving state by time is ; its probability of occupying state is for unequal rates.
Transition intensity matrix Created 2026-10-06 Updated 2026-11-05
For a finite-state continuous-time Markov chain, a transition intensity matrix has nonnegative off-diagonal rates and . Every row sums to zero. In the time-homogeneous case its transition probability matrix is the matrix exponential . An absorbing state has a zero row.
Uniform decreasing Markov chain Created 2026-10-05 Updated 2026-10-06
On , make an absorbing state and let each state move uniformly to . Its expected hitting time of , allowing zero time when started at , is . First-step analysis gives ; subtracting consecutive recurrences yields and hence the formula.
A finite stock decreases by choosing a uniformly random positive number of its remaining items at every step. The remaining stock is a Markov chain with for and . State is an absorbing state, and every positive state is a transient state. If denotes the expected number of positive-stock steps before absorption, a first-step decomposition gives and . Subtracting consecutive equations yields , so , the th harmonic number.
Voter model 2026-10-14
At each site, a rate-one clock rings and the site copies the state of a randomly chosen neighbor. The model is a Markov process on Ising spin configurations, interpreted through its local update infinitesimal generator. All constant configurations are absorbing state. Tracing copying arrows backwards gives the voter-model duality with coalescing random walks.