For a Markov chain started at state , let denote the first positive return time, using time for the starting state. A recurrent state has ; a transient state has return probability strictly less than . An absorbing state satisfies , so the chain remains there forever after arrival and it is necessarily a recurrent state.
State is an absorbing state and a recurrent state. From every positive state, the stock decreases strictly at the next step and can never increase, so a return to that positive state is impossible. Thus all positive states are transient, with return probability zero. Starting from , absorption occurs within at most sticker-taking children.
An absorbing state has ; once reached, the chain never leaves it. For a nonabsorbing initial state, condition on its first transition and use the Markov property:Here and . The other equations areSolving givesFor absorption into either state one or four, let be the expected number of steps, with . first-step analysis givesHence andThe expectation is finite because each transient two-step cycle has probability at most of remaining transient; the survival tail is geometrically bounded. This also justifies solving the expectation equations without assuming finiteness in advance.
A state is a recurrent state if the probability of a return at some positive time, starting there, is one. It is a transient state if that return probability is less than one. It is an absorbing state if its one-step probability of remaining there is one. An absorbing state is recurrent, since every subsequent time is a return.
The absorbing states give . Multiplying the difference equation in (i) by givesThe second finite difference of is , so a particular solution is ; the homogeneous solutions are . Imposing the two boundary conditions gives . ThereforeUniqueness follows because the difference of any two solutions has zero second finite difference, hence is affine function, and vanishes at both endpoints. This is the mean absorption time of a lazy symmetric random walk with holding probability .
Let . By the law of total probability,There is no return contribution from the absorbing state . Since can be entered only from , for . Substitution yieldsNo independence between successive positions is assumed; the conditional transitions give the recurrence.
Progressive illness-death model 2026-10-07
This continuous-time multi-state model permits progression , direct death , and death after progression , with no recovery and death an absorbing state. For constant positive rates , write . Integrating over the time of the progression arrow givesUse when . The other transition probabilities are , , , and . Unlike a purely sequential irreversible three-state disease model, the direct-death rate contributes to the first-state exit rate. A mixed panel and exact-death likelihood accounts for panel visits and exact death times with different kinds of factors.
Three-state irreversible disease model 2026-10-06
A sequential continuous-time Markov chain has transitions with rates and an absorbing state . Its probability of ever leaving state by time is ; its probability of occupying state is for unequal rates.
Transition intensity matrix Created 2026-10-06 Updated 2026-11-05
For a finite-state continuous-time Markov chain, a transition intensity matrix has nonnegative off-diagonal rates and . Every row sums to zero. In the time-homogeneous case its transition probability matrix is the matrix exponential . An absorbing state has a zero row.
Uniform decreasing Markov chain Created 2026-10-05 Updated 2026-10-06
On , make an absorbing state and let each state move uniformly to . Its expected hitting time of , allowing zero time when started at , is . First-step analysis gives ; subtracting consecutive recurrences yields and hence the formula.
Uniform depletion Markov chain 2026-10-31
A finite stock decreases by choosing a uniformly random positive number of its remaining items at every step. The remaining stock is a Markov chain with for and . State is an absorbing state, and every positive state is a transient state. If denotes the expected number of positive-stock steps before absorption, a first-step decomposition gives and . Subtracting consecutive equations yields , so , the th harmonic number.
Voter model 2026-10-14
At each site, a rate-one clock rings and the site copies the state of a randomly chosen neighbor. The model is a Markov process on Ising spin configurations, interpreted through its local update infinitesimal generator. All constant configurations are absorbing state. Tracing copying arrows backwards gives the voter-model duality with coalescing random walks.