Codex @codex  0

A Markov chain with at least one absorbing state, such that from each state an absorbing state can be reached. In a finite such chain, absorption occurs with probability 1. Expected times to absorption satisfy a linear system obtained by conditioning on the first transition.

For a finite, time-homogeneous Discrete-time Markov chain with row-stochastic transition matrix $P$, a state $a$ is absorbing exactly when $P_{aa}=1$. Nonnegativity and the row sum then force $P_{aj}=0$ for $j\ne a$: once entered, $a$ is never left. Equivalently, $\{a\}$ is a closed communicating class. A larger closed communicating class may keep moving internally; closure means confinement to a set, whereas absorption at a state means remaining at one point.

An absorbing state is useful for recording a terminal outcome: after a toy routine finishes, keep its outcome in the state space so that the same transition matrix still describes subsequent steps. Calling a finite Markov chain absorbing usually also requires that some absorbing state be accessible from every state. Merely containing an absorbing state is insufficient: another closed communicating class could prevent absorption. See Grinstead and Snell, Introduction to Probability, section 11.2 for this convention.

Consider a constructed routine with unfinished modes $u,v$ and terminal outcomes $A,B$. All the following transition probabilities are chosen for illustration, not estimated from observations. In state order $(u,v,A,B)$, takeThe communicating class $\{u,v\}$ is open, while $\{A\}$ and $\{B\}$ are closed communicating classes consisting of absorbing states. Both terminal outcomes are accessible from both unfinished modes, sometimes by an indirect path.

Let $T=\inf \{n\ge 0:X_n\in \{A,B\}\}$ be the terminal hitting time. Write $h_i={\mathbb{P}}_i(X_T=A)$ for the absorption probability and $t_i={\mathbb{E}}_{i}T$ for the expected hitting time, with the subscript denoting the starting state. First-step analysis givesHere $h_A=1$, $h_B=0$, and $t_A=t_B=0$. Solving these two pairs of linear equations yieldsStarting at $u$ therefore ends at $A$ with probability $0.6$ and takes $3.2$ steps on average to reach either terminal state. The terminal hitting time counts the transition into the outcome, but no later self-transitions. Outcome and duration answer different questions: from $v$, outcome $A$ is less likely, yet termination is faster on average.

The matrix power $P^2$ gives the probability distribution after two steps from $u$ asThus ${\mathbb{P}}_u(T>2)=1/2$, and the probability of having already reached $A$ is $3/8$, still below its eventual absorption probability $3/5$. From either unfinished mode, each next step has terminal probability at least $1/4$. Consequently ${\mathbb{P}}_i(T>n)\le (3/4{)}^n$, proving termination almost surely and finite expected hitting time directly. The Fundamental matrix of a finite absorbing Markov chain packages both calculations and counts visits to unfinished modes.

For a discrete-time Markov chain and a set $A$ of absorbing states, its absorption time is the hitting time $T_A=\inf \{m\ge 0:X_m\in A\}$. It is zero when the initial state already lies in $A$, and infinite on a sample path that never enters $A$. Finiteness almost surely need not imply a finite expected value. For a finite-state model, a uniform positive probability of absorption within a fixed number of steps gives a geometric series bound on the survival probabilities and proves that the mean is finite. The mean absorption time of a lazy symmetric random walk is an explicit example.

In a finite, time-homogeneous Discrete-time Markov chain with row-stochastic transition matrix $P$, write $i\to j$ when $j$ is accessible from $i$, meaning $(P^n{)}_{ij}>0$ for some $n\ge 0$. States communicate, written $i\leftrightarrow j$, when both $i\to j$ and $j\to i$. Communication is an equivalence relation, and a communicating class is an entire equivalence classThus communicating classes form a partition of the state space; a smaller mutually communicating subset is not itself a class. See Maria Cameron's Markov Chains notes, section 1.2.

The allowance of zero steps matters: $P^0=I$, the identity matrix, makes $i\to i$ true even for a state with no possible positive-time return. Symmetry is built into communication. For transitivity, concatenate a positive-probability path from $i$ to $j$ with one from $j$ to $k$, and do the same in reverse. These observations prove the equivalence relation directly. One-way accessibility alone is not symmetric and does not define the classes; see Accessible state of a Markov chain.

A closed communicating class satisfies $P_{ij}=0$ for every $i\in C$ and $j\notin C$. Once entered, it cannot be left. A singleton class is an absorbing state exactly when it is closed. An open communicating class has an outgoing transition; after a path leaves it, it cannot return, since that would make the visited outside states communicate with it. The internal motion of a closed class can nevertheless continue forever.

For this constructed example, in state order $(s,c_1,c_2,a)$, the transition matrix isThere are exactly three communicating classes: $\{s\}$, $\{c_1,c_2\}$, and $\{a\}$. Only the first is open. The second is closed but neither of its states is an absorbing state; they alternate deterministically. Although $s\to c_2$ via $c_1$, the reverse accessibility fails. A direct matrix power check gives $(P^2{)}_{s,c_2}=1/4$ and $(P^2{)}_{c_2,s}=0$.

Starting at $s$, let $\tau$ be the first hitting time of either closed class. Staying at $s$ for $n$ steps has probability $2^{-n}$, so ${\mathbb{P}}_s(\tau >n)=2^{-n}$ and ${\mathbb{E}}_s\tau ={\sum }_{n\ge 0}{2}^{-n}=2$. The two eventual destination classes each have probability ${\sum }_{k\ge 0}(1/2{)}^k(1/4)=1/2$. Hence the hitting probability of the absorbing state $a$ is only $1/2$, and its unconditional expected hitting time from $s$ is infinite: on half the paths it is never hit. In particular, entering a closed class with certainty does not imply entering an absorbing state with certainty.

Classifying the state space separates two questions: which closed region will eventually be entered, and how does the Markov chain move inside that region? In this example the restriction to $\{c_1,c_2\}$ is an irreducible Markov chain with period two; the restriction to $\{a\}$ is a one-state irreducible Markov chain with period one. The whole Markov chain is reducible. The absorbing state example illustrates a different possibility: every closed class is a singleton, so eventual class selection is also terminal outcome selection.

For a Galton-Watson process with one ancestor, the branching extinction probability is the probability that some generation is empty:The equality uses the absorbing state at zero. The complementary branching survival probability $1-q$ is the probability that every generation is nonempty. The Galton-Watson extinction fixed point computes $q$ from the offspring distribution.

For $p_0=p_1=1/4$, $p_2=1/2$, one ancestor has $q=1/2$. Starting instead from $m$ ancestors with independent descendant families givesAll $m$ families must die out. With three ancestors, eventual extinction has probability $1/8$ and survival has probability $7/8$; by generation two, extinction has probability $(11/32{)}^3=1331/32768\approx 0.040619$. Extra ancestors change the extinction probability without changing the offspring distribution. ^[ref]

For finite offspring mean $\mu$, the extinction classification is:

The critical exception is $p_1=1$: every individual has exactly one child, $F(s)=s$, and $Z_n=1$ forever from one ancestor. Thus $\mu =1$ but $q=0$; a blanket claim that every critical Galton-Watson process becomes extinct would be false. These cases concern the Galton-Watson process with its common offspring distribution and independence assumptions. ^[ref]

This probability answers whether any lineage persists, which the offspring mean alone cannot quantify in the supercritical case. A branching process conditioned on extinction uses the same fixed point to reweight reproduction and describe families known eventually to die out.

Take a birth-death chain on $\{0,1,2,3,4\}$. At each interior state, move up one state with probability $p=1/2$ and down one state with probability $q=1/2$. The endpoints are absorbing states: $P_{00}=P_{44}=1$. This is a small gambler's ruin model in which the state can represent capital, and reaching $4$ is success.

Let $T_a=\inf \{n\ge 0:X_n=a\}$ be the hitting time of endpoint $a$, and let $T=\min (T_0,T_4)$ be the first endpoint hitting time. The desired hitting probability is $h_i={\mathbb{P}}_i(T_4<T_0)$. First-step analysis and the Markov property giveThe interior equation says adjacent differences are equal. Four equal increments must bridge the boundary values $0$ and $1$, soFor example, from state $2$ the hitting probability is $1/2$, and its one-step average is $\frac{1}{2}(1/4)+\frac{1}{2}(3/4)=1/2$. Individual sample paths can wander back and forth, but the same averaging rule already accounts for all of them.

Now ask how long it takes to reach either endpoint. The expected hitting time $t_i={\mathbb{E}}_{i}T$ counts steps, including the transition that first reaches an endpoint. First-step analysis givesThis is a discrete Poisson equation, $(I-P)t=1$ in the interior. The extra $1$ counts the step just taken. The quadratic valuessatisfy the equations; from state $2$, $t_2=1+\frac{1}{2}(3)+\frac{1}{2}(3)=4$ steps. The hitting probability is harmonic, whereas the expected hitting time has a nonzero source term. These are special cases of expected duration of symmetric gambler's ruin; see Hao Wu's MIT lecture 16, gambler's ruin, slide 9.

For a biased random walk with $p=0.6$ upward and $q=0.4$ downward, keep the same absorbing states. The interior harmonic equation becomes $h_i=p{h}_{i+1}+q{h}_{i-1}$. Using biased gambler's ruin probability with these boundary values givesThe three interior hitting probabilities are approximately $0.415385$, $0.692308$, and $0.876923$. Thus starting at $2$ now succeeds about $69.23\%$ of the time. A direct check is $0.6(57/65)+0.4(27/65)=9/13$. The curved profile reflects the unequal transition weights: its successive increments shrink by the factor $q/p=2/3$. The formula, with the endpoint denoted $T$ there, is verified in Karger and Lynch's MIT lecture 26, sections 2.3-2.4; the five-state numbers here are direct substitutions.

Why does harmonic averaging produce the correct hitting probability? With the natural filtration ${\mathcal{F}}_n=\sigma (X_0,\dots ,X_n)$, the Markov property gives the conditional expectation identityFor a function harmonic everywhere, this makes $h(X_n)$ a martingale whenever it is integrable. For a function harmonic only in the interior, $M_n=h(X_{n\wedge T})$ is a stopped martingale. In either five-state example, $0\le M_n\le 1$. Also, from any interior state at most three consecutive upward steps reach $4$, with probability at least $p^3>0$. Repeated three-step blocks give ${\mathbb{P}}_i(T>3k)\le (1-p^3{)}^k$, so $T$ is almost surely finite and has finite expected value; compare finite-interval exit times have geometric tails. Applying optional stopping at $n\wedge T$ and then the bounded convergence theorem givesThe boundedness and finite hitting time justify this passage to the limit. See Steven Lalley's notes, harmonic functions and stopped expectations, theorem 1, and stopped martingale in discrete time for this bounded version of optional stopping.

Uniqueness has a short maximum principle explanation. On a finite state space, the difference of two harmonic extensions with the same boundary conditions is zero on the boundary. If its positive maximum occurs inside, the weighted average forces every successor with positive transition probability to have the same maximum. A path to the boundary propagates that maximum to zero, a contradiction. Applying the argument to the negative of the difference proves uniqueness. Reachability of the boundary is essential: an interior closed communicating class can retain an arbitrary constant.

// Target: markov-process.bigb

For a finite Markov chain in which an absorbing state is accessible from every state, order the nonabsorbing states first and writeHere $I$ is the identity matrix and $N$ is the matrix inverse of $I-Q$. With $T$ the first hitting time of the absorbing set,Thus $N$ gives the expected occupation count before absorption, including the initial visit; $B_{ia}$ is the absorption probability of state $a$; and $t_i$ is the expected hitting time of the absorbing set. The vector $1$ has every entry one. See Grinstead and Snell, section 11.2, theorems 11.4-11.6.

For the toy routine in absorbing state, direct matrix multiplication givesThe first row of $N$ says that starting at $u$ gives $2.4$ visits to $u$ and $0.8$ to $v$ before termination. Their sum is the mean duration $3.2$. The identities $(I-Q)N=I$, $B=QB+R$, and $t=1+Qt$ independently verify the inverse, outcome, and duration calculations. The absorbing set must be reachable from every state for these finite formulas: applying them to all nonabsorbing states in the closed-cycle example would leave a persistent cycle in $Q$ and make $I-Q$ singular.

For a Galton-Watson process with one ancestor and offspring probability generating function $F$, the branching extinction probability $q$ is the smallest fixed point of $F$ in $[0,1]$. A direct way to find it is to compute extinction by generation $n$:Zero is an absorbing state, so extinction by generation $n$ is exactly the event $Z_n=0$. ^[ref]

For $F(s)=1/4+s/4+s^2/2$, the equation isBoth $1/2$ and $1$ are fixed points, but iteration from zero selects $q=1/2$. The first values are $q_1=1/4$, $q_2=11/32$, $q_3=809/2048\approx 0.395020$, and $q_6\approx 0.462186$. Iteration started at $1$ would stay at $1$ forever and would give the wrong extinction answer for this offspring distribution.

The vertical steps in the figure apply the offspring probability generating function; horizontal steps return to the diagonal to start the next iteration. This gives a useful numerical calculation even when solving $F(s)=s$ explicitly is difficult. For the displayed law, every finite iteration is a lower bound for the eventual branching extinction probability; at generation six, about $0.037814$ additional extinction probability remains beyond the current empty families.

Why the smallest fixed point? The ancestor's children's descendant families are independent; given $k$ children, all are empty $n$ generations later with probability $q_n^k$. Averaging gives $q_{n+1}={\sum }_k{p}_k{q}_n^k=F(q_n)$. The increasing extinction events give the limit $q$, and continuity of the probability generating function yields $F(q)=q$. For any other fixed point $r\in [0,1]$, $q_0\le r$ and monotonicity of $F$ imply $q_n\le r$ at every step; hence $q\le r$. ^[ref]

A Galton-Watson process is a branching process that counts individuals by generation. Start with one ancestor, $Z_0=1$, and let all offspring counts ${\xi }_{n,j}$ be independent and identically distributed random variables taking nonnegative integer values with a common offspring distribution. ThenThe empty sum is zero: zero is an absorbing state, so once a generation is empty every later generation is empty. The same offspring distribution applies to every individual in every generation. ^[ref]

For a concrete offspring distribution, take $p_0=1/4$, $p_1=1/4$, $p_2=1/2$, and $p_k=0$ for $k\ge 3$. An individual has no children with probability $1/4$, one with probability $1/4$, and two with probability $1/2$. The offspring expected value is $\mu =5/4$. A possible sequence of generation sizes is $1,2,1,0,0,\dots$: two children need not lead to a permanently growing family.

The probability generating function $F(s)=1/4+s/4+s^2/2$ gives the exact generation probability distributions through branching-process generating-function iteration. With $G_0(s)=s$ and $G_{n+1}(s)=F(G_n(s))$, the second generation hasThus $\mathbb{P}(Z_2=0)=11/32$ and $\mathbb{E}[Z_2]=25/16$. The figure shows full generation probability mass functions, computed by polynomial function composition, through generation six; no simulated frequencies are used.

The contrast is striking: the expected generation size in a Galton-Watson process grows as $(5/4{)}^n$, while the branching extinction probability is $1/2$. By generation six, $\mathbb{P}(Z_6=0)\approx 0.462186$ although $\mathbb{E}[Z_6]\approx 3.814697$. Larger surviving families carry the rising expected value; that mean is an average over possible families, not a forecast of one family's path. The Galton-Watson extinction fixed point turns an unbounded number of future generations into a single equation for this probability.

The historical question was the disappearance of surnames passed through male descendants. In Watson and Galton, On the Probability of the Extinction of Families, pages 139–140, Francis Galton formulates a constant-law model counting sons who reach adulthood, and Watson computes later generation probability distributions by composing generating functions. Those are assumptions of their model, rather than a claim that actual human reproduction follows this offspring distribution.

A harmonic function assigns a value to each state of a Markov chain such that one transition leaves its expected value unchanged. This turns a question about many possible future sample paths, such as which endpoint is reached first, into a local averaging rule. For a transition matrix $P$, a real-valued function $h$ is harmonic at state $i$ whenHere ${\mathbb{E}}_i$ means expected value conditional on starting at $X_0=i$; on a countable state space the sum must be absolutely convergent. If this holds at every state, write $Ph=h$. At an absorbing state the equation is automatic, so prescribed boundary conditions supply information that the averaging rule cannot.

For a finite Markov chain, choose a boundary set $B$ that can be reached from every state, and assign boundary values $b$. If $T_B$ is its first hitting time, the harmonic extension isChoosing $b$ to be the indicator function of one target within $B$ gives a hitting probability. This is a discrete Dirichlet problem: enforce the averaging rule outside $B$ and fix the values on $B$. See Hao Wu's MIT lecture 8, harmonic extension, slide 5. The construction extends the mean-value idea of a harmonic function; for a random walk on a graph it is a discrete harmonic function.

In a nearest-neighbor voter model on the square lattice, any two ancestral coalescing random walks meet almost surely. For an invariant probability measure of a Markov process, the disagreement probability is constant in time but bounded by the vanishing probability of noncoalescence. Hence every pair of sites agrees almost surely. Countability concentrates the measure on the two constant configurations; both are absorbing state, so every mixture of their Dirac measures is invariant. Translation invariance need not be assumed.

For a finite, time-homogeneous Discrete-time Markov chain with row-stochastic transition matrix $P$, irreducibility meansEquivalently, all states form one communicating class. The step count may depend on $i,j$; irreducibility does not require every one-step transition probability to be positive. Drawing an arrow $i\to j$ whenever $P_{ij}>0$ makes the condition exactly that of a strongly connected directed graph. It is a property of the entire chosen state space, independent of the initial probability distribution.

Irreducibility describes where the Markov chain can travel; aperiodicity describes the possible lengths of return paths. For example, restricting the second diagram in communicating class to its closed pair $\{c_1,c_2\}$ giveswhere $I$ is the identity matrix. This Markov chain is irreducible, but each state's period of a state in a Markov chain is two. Every destination is reachable, yet a given destination cannot be reached at every sufficiently large step count. Starting at $c_1$ produces alternating one-time probability distributions. See Hao Wu's MIT lecture 2, irreducibility and periods for the definitions.

Adding a holding probability of a Markov chain $\epsilon$ at each state, with $0<\epsilon <1$, givesThe arrows between the two states remain, so the Markov chain remains irreducible. A positive one-step return makes it an aperiodic Markov chain. At $\epsilon =1/2$, both rows are $(1/2,1/2)$ and $C_{1/2}^n=C_{1/2}$ for every $n\ge 1$: the initial state is forgotten after one step. At $\epsilon =1$, however, both states become absorbing states, giving two separate communicating classes and losing irreducibility.

For a finite Markov chain, irreducibility guarantees a unique stationary distribution with positive weight at every state. Adding aperiodicity guarantees convergence ${\mu }_0{P}^n\to \pi$ for every initial probability distribution; see Hao Wu's MIT lecture 4, convergence theorem. The Markov chain ergodic theorem gives convergence of empirical visit frequencies even for an irreducible periodic Markov chain. These results explain why checking connectivity is a useful first step before studying equilibrium. Conversely, any absorbing state prevents irreducibility if there is another state: it cannot reach that state. The one-state Markov chain is the exception.

Consider a random walk on $\{0,1,\dots ,n\}$ with absorbing states $0,n$. At each interior state it stays in place with probability $q$, and moves one position in either direction with probability $(1-q)/2$, where $0\le q<1$. Its mean absorption time from $k$ isProof. A sequence of at most $n$ consecutive leftward moves ensures absorption, with probability at least $((1-q)/2{)}^n>0$. Dividing time into blocks of length $n$ gives a geometric series bound on the survival probabilities, so the expected value is finite. Let $e_k={\mathbb{E}}_k[T]$. The law of total expectation and first-step analysis giveThus $e_{k+1}-2e_k+e_{k-1}=-2/(1-q)$. A particular solution of this linear recurrence relation is $-k^2/(1-q)$; its homogeneous solutions are $A+Bk$. The two boundary conditions give $A=0$, $B=n/(1-q)$, proving both the formula and its uniqueness. Holding slows the usual symmetric random walk by the factor $1/(1-q)$.

A clinic-observed interval ending in a recorded state contributes a transition probability. If the next observation is exact entry into an absorbing state but the preceding state is unobserved, its statistical probability density is ${\sum }_{s\text{ living}}{p}_{rs}(t)q_{sD}$, not $p_{rD}(t)$. Multiplying these conditional contributions yields a likelihood for mixed panel and exact-event observations.

Use $T_y=\inf \{n\ge 0:X_n=y\}$ and the positive return time $T_x^{+}=\inf \{n\ge 1:X_n=x\}$. The walk has transitions $p_{uv}=c_{uv}/d(u)$, $d(u)={\sum }_v{c}_{uv}$; its transition matrix defines a reversible Markov chain, although the transition probabilities need not be symmetric.

The voltage $h(u)=P_u(T_x<T_y)$ equals one at $x$, zero at $y$, and is a harmonic function on a graph away from the terminals: $h(u)={\sum }_v{p}_{uv}h(v)$. For this unit electric potential difference the electric current isThe first-step analysis therefore proves the return probability from effective resistance identity:Finite irreducibility ensures eventual hitting of one of the terminals, so no missing escape event appears in this finite-network calculation.

For an infinite locally finite connected graph, exhaust it by finite sets and wire their exteriors to one sink. The event of exiting before the first return decreases to the event of never returning. Thuswith $1/\infty =0$. In particular recurrence is equivalent to infinite effective resistance to infinity.

After deleting an edge, consider any infinite component and choose a root $o$ away from that edge's endpoints. Its vertex degree $d(o)$ is unchanged. On every finite wired exhaustion the Rayleigh monotonicity principle makes the resistance no smaller; the limit of a sequence does too. A recurrent original network consequently leaves a recurrent component. Finite components give recurrent walks as well, with an isolated graph vertex viewed as an absorbing state. This proves that edge deletion preserves recurrence. The phrase “no less recurrent” refers to this recurrence comparison: at a deleted-edge endpoint the vertex degree factor also changes, so monotonicity of its first-return probability alone should not be inferred.

In the voter model on ${\mathbb{Z}}^2$, each site has a rate-one Poisson process clock; when it rings, that site copies one of its four neighbors, chosen uniformly. States are zero or one. For a cylinder function $f$, the local infinitesimal generator iswhere ${\eta }^{x\leftarrow y}$ agrees with $\eta$ except that the voter model opinion at $x$ is replaced by the voter model opinion at $y$. Only sites in the support of $f$ contribute, so this infinitesimal generator sum is finite.

The voter-model duality comes from independent Poisson processes of copying arrows of rate $1/4$ on each ordered neighboring pair. Trace the ancestors of specified sites backwards from time $t$: each ancestral line is a rate-one continuous-time random walk with uniform nearest-neighbor steps, and lines coalesce when they meet. The voter model opinions at time $t$ are the initial voter model opinions at those ancestral locations. This construction also explains the general duality theorem being used.

Two ancestral walks on ${\mathbb{Z}}^2$ meet almost surely. Until their meeting, their difference is a rate-two symmetric nearest-neighbor walk, and simple random walk in two dimensions is recurrent. If ${\tau }_{xy}$ is their meeting time, this givesfor every initial law $\nu$. If $\nu$ is invariant, the left side is the fixed probability $\nu (\eta (x)\ne \eta (y))$, so it is zero. This holds for every pair. Countability then implies that $\nu$ is concentrated on configurations with all voter model opinions equal. Both constant configurations are absorbing state. Thus the invariant measures of the two-dimensional voter model are exactlyNo translation-invariance assumption on $\nu$ was used.

For the last assertion, let the initial set of one-valued opinions $F$ be finite. Single-line duality givesEach transition probability tends to zero. One direct verification is the Fourier transform for the rate-one walk:The integrand has modulus at most one and tends to zero except at the single point $k=0$; dominated convergence proves the claim. For any finite observation set $K$, the union bound now givesTherefore every finite voter model opinion pattern converges in law to the all-zero pattern. These cylinder distributions determine weak convergence of probability measures in the compact product topology, proving finite-seed local extinction in the voter model and

The permitted transitions form the following diagram. Each arrow is labelled by its fitted transition intensity in units of inverse years; the absence of an arrow is a structural restriction, not an estimated small effect.

State 1 can move directly to 2, 3 or 5; state 2 can move to 4 or 5; state 3 can move to 4 or 5; and state 4 can move only to 5. There are no recovery transitions, and state 5 is an absorbing state. The rates of leaving the four living states are the sums of their outgoing arrows:The diagonal generator entries are their negatives, so every row of $Q$ sums to zero. These diagonal entries are not arrows or negative event rates. For example the direct $1\to 5$ rate is $0.004$ per year, while death after reaching another state can occur at a much larger rate. The printed fit has no covariates on the intensities, so it estimates one pooled rate system and contains no treatment-effect comparison, despite the trial's treatment randomization. The reported $-2\ell$ describes the fitted likelihood level; by itself it is not a goodness-of-fit test without a reference or competing model.

For a homogeneous continuous-time Markov chain, the holding time in a living state $r$ before the next transition has survival probability $e^{-{\lambda }_{r}t}$, where ${\lambda }_r={\sum }_{s\ne r}{q}_{rs}=-q_{rr}$. Indeed, conditional on still occupying that state, the probability of leaving in the next $dt$ is ${\lambda }_{r}dt+o(dt)$, so its survival function satisfies $S^{\prime }(t)=-{\lambda }_{r}S(t)$ and $S(0)=1$. Integrating gives the mean holding time from a transition intensity matrix:This is the mean time until any allowed exit from that state, not necessarily the patient's remaining lifetime.

For confidence limits, use the supplied standard error $s_r$ of the corresponding diagonal generator estimate. Changing its sign does not change its standard error, so it is also the standard error of ${\hat{\lambda }}_r$. The delta method, with derivative $d(1/\lambda )/d\lambda =-1/{\lambda }^2$, givesThe resulting approximate 95% Wald confidence intervals areThe diagonal standard errors already incorporate the covariance among outgoing rates. They should not be replaced by the square root of a sum of squared off-diagonal standard errors, which would silently assume those rate estimates independent.

Alternatively construct a positive rate confidence interval and apply the confidence interval for an inverse rate transformation. Here every normal rate interval has a positive lower endpoint, givingThese approximate intervals are, respectively, $(10.863,12.830)$, $(4.497,7.161)$, $(2.397,3.117)$ and $(1.548,2.631)$ years. They are asymmetric because inversion is nonlinear; positive log-rate or likelihood-based intervals could also be inverted. The symmetric delta intervals and the inverted-rate intervals are two legitimate first-order approaches and need not have numerically identical endpoints.

The death state has zero exit rate, so its holding time is infinite. There is no finite mean-sojourn estimate or ordinary finite confidence interval for this structurally absorbing state. All intervals above concern the mean parameter, not a prediction interval for one patient's realized holding time.

The transition matrix over five years is $P(5)=e^{5Q}$. Its first row conditions on starting in state 1. The displayed approximate probabilities mean that, five years later, about 65% remain disease-free, 6% have local recurrence only, 10% have distant metastasis only, 2% have both, and 16% have died. They are probabilities of the states occupied at five years; they are not the probabilities of the next jump destinations or annual transition intensities. In particular the positive probability of state 4 does not require a direct $1\to 4$ arrow: the paths $1\to 2\to 4$ and $1\to 3\to 4$ contribute to it. The probability of death includes deaths after any allowed sequence of intermediate states.

The printed row sums to $0.99$ rather than one because the entries are approximate; an exact stochastic transition row sums to one. Exponentiating the displayed three-decimal generator gives approximately $(0.65377,0.06490,0.09775,0.02532,0.15826)$, illustrating that the edited, low-precision table should not be treated as an exact matrix exponential of its separately rounded rates.

State 5 is an absorbing state, so once entered it can never be left. Its generator row is zero, and the last row of every positive-time transition matrix is thereforeThis explains all four leading zeroes in the printed last row.

The modified transition matrix $\tilde{P}$ describes the original Markov chain until it hits $B$, and then keeps it at the first state hit. Each state of $B$ is therefore an absorbing state, andFor a nonnegative superharmonic function for a Markov chain $g$ satisfying the boundary condition, part (c) and positivity of the transition matrix give by induction $g\ge {\tilde{P}}^{n}g$. Also $g\ge 1_B$. ConsequentlyLet $n\to \infty$ to obtain $g_i\ge h_i$. Since $h$ satisfies the conditions by parts (a),(b), $h$ is their pointwise minimal solution. This extends the hitting probability is the minimal nonnegative harmonic extension principle to the superharmonic inequality. Nonnegative sums justify the argument on a countable state space as well as a finite one.

Now suppose the Markov chain is irreducible and recurrent, and let $g$ satisfy condition (a). In such a chain every state is hit almost surely from every other state. To see this directly, recurrence gives infinitely many visits to a starting state $i$. Irreducibility supplies a finite path from $i$ to any target $j$ with positive probability $\alpha$, chosen with no return to $i$ before reaching $j$. At each successive return to $i$, the Strong Markov property gives another trial with conditional success probability at least $\alpha$. The chance of avoiding $j$ through $N$ such trials is at most $(1-\alpha {)}^N$, which tends to zero.

Apply the absorbing modification at the singleton $\{j\}$. Part (c) holds for arbitrary $g$ satisfying (a), soSince $g_j\ge 0$ and the probability tends to one, $g_i\ge g_j$. Interchanging $i,j$ yields equality. Thus every nonnegative superharmonic function is constant on an irreducible recurrent Markov chain.

Conversely, if an irreducible Markov chain is not recurrent, choose a state $j$ with return probability $r_j={\mathbb{P}}_j({\tau }_j^{+}<\infty )<1$. Its hitting probability $h$ for $\{j\}$ satisfies condition (a), and $h_j=1$. The first-step formula for a return after time zero gives $(Ph{)}_j=r_j<1$. Therefore $h$ cannot be constant: a constant function with value one at $j$ would have $(Ph{)}_j=1$. This supplies a nonconstant nonnegative superharmonic solution. HenceThe admissible constant solutions are precisely the nonnegative constants. This is recurrence characterized by constant nonnegative superharmonic functions.