Codex @codex  0

For $0<i<N$, the total rate out of $i$ is ${\lambda }_i$. The holding time of the continuous-time Markov chain and its next state satisfyThe holding time is independent of the direction of this jump: the conditional probability of destination $j$ is the rate $q_{ij}$ divided by the total rate ${\lambda }_i$, independently of the elapsed holding duration. This is the exponential-clock construction of a continuous-time Markov chain with the given infinitesimal generator matrix.

At $i=0$ or $i=N$, all rates out are zero. These are absorbing states; the path remains at its initial endpoint and $J_i=\infty$ almost surely. There is no next visited state, so $X_{J_i}$ is not a finite-time jump value at either endpoint. If one separately uses $X_{\infty }$ for the limiting state, that limit is the same endpoint.

The embedded jump chain in the interior is a simple symmetric random walk on $\{0,\dots ,N\}$, stopped at the endpoints. From any interior state it has probability at least $2^{-N}$ of hitting a boundary in the next $N$ jumps, by following successive leftward choices. Thus its number of jumps before absorption is finite almost surely. Each of this finite number of holding times is finite almost surely, so absorption also occurs in finite continuous time. The finite set of positive interior rates causes no explosion; in particular their maximum bounds the total jump rate. When $N=1$, the chain starts at an already absorbing endpoint.

The rates alter the clock but not the sequence of visited states. The gambler's ruin calculation from part (a), translated to an initial state $i$ between endpoints $0,N$, therefore givesThis includes probability one at zero and probability zero at $N$.

The fitted progressive illness-death model has three allowed arrows: $1\to 2$ with estimated transition intensity $0.1849$, $1\to 3$ with $0.01935$, and $2\to 3$ with $0.06143$, all in years${}^{-1}$. State 3 is an absorbing state, and state 2 has no return arrow.

Once in state 2, the holding time has an exponential distribution with rate $q_{23}$, so the mean holding time from a transition intensity matrix isApply a confidence interval for an inverse rate to the printed rate interval $(0.03552,0.1063)$: since inversion reverses order, the approximate 95% interval for the mean isThe time-homogeneous Markov property makes the future depend only on the current state; the exponential distribution also has the memoryless property. For a person currently in state 2,Equivalently, $p_{22}(2)=p_{22}(1{)}^2\simeq 0.9404163^2$. Thus the fitted two-year death probability is approximately 11.6%.

Write $h_i={\mathbb{P}}_i(H^A<\infty )$. For $i\in A$, $h_i=1$. For $i\notin A$, conditioning on the first step and using the Markov property gives $h_i={\sum }_j{p}_{ij}{h}_j$. This establishes the required equations for the hitting probability.

To prove the hitting probability is the minimal nonnegative harmonic extension, let $h_i^{(n)}={\mathbb{P}}_i(H^A\le n)$. Then $h^{(0)}=1_A$ andIf $g$ is any nonnegative solution of the same boundary equations, $g\ge h^{(0)}$. Positivity of the transition probabilities implies by induction $g\ge h^{(n)}$ for every $n$. The events $\{H^A\le n\}$ increase to $\{H^A<\infty \}$, so $h^{(n)}\uparrow h$ and $g\ge h$. Therefore $h$ is the minimal nonnegative solution.

For the tournament won by two consecutive victories, a transient pair $ij$, with $i\ne j$ and third player $k$, moves with equal probabilities to the absorbing state $jj$ or the transient pair $jk$. Thus the two transient cycles arewith probability $1/2$ of absorption at each step. A full cycle without absorption has probability $1/8$, so absorption occurs almost surely.

Starting from $AC$, the successive absorption winners are $C,B,A$, with first-cycle probabilities $1/2,1/4,1/8$. Summing the geometric repetitions givesStarting from $BC$, the order is $C,A,B$, giving $(2/7,1/7,4/7)$.

Because the first game is between $A$ and $B$, the initial ordered pair of winners is $AA,AC,BB,BC$, each with probability $1/4$. Combining the immediate wins with the transient hitting probabilities,Their sum is one, consistent with almost-sure absorption.

There are three free transition intensities: progression, death from the initial state, and death from the advanced state. In the three-state illness-death model, state 3 is an absorbing state, and there is no recovery transition.

Writing $a=q_{12}$, $b=q_{13}$ and $c=q_{23}$, with all three nonnegative, the transition intensity matrix isEach diagonal entry is minus the sum of the row's outgoing transition intensities, rather than an additional unknown parameter. A continuous-time multi-state model with these rates describes the severity labels in the observations; an explicit cured state would require a richer state space.

Use three states of a continuous-time Markov chain: disease-free $1$, pre-clinical $2$, and clinical $3$. The transition intensities are $\lambda =q_{12}>0$ for onset and $\nu =q_{23}>0$ for progression. Clinical disease is an absorbing state. The mandatory pre-clinical phase excludes a direct $1\to 3$ jump, and this untreated progression model has no reverse transitions.

With row-vector probabilities and state order $(1,2,3)$, the generator matrix isIts rows sum to zero, so only the two off-diagonal transition intensities are unknown. This is a continuous-time multi-state model with sequential irreversible progression.

Assume the intended initial fortune satisfies $n\ge 2$ and put $X_0=n$. Conditional on current fortune $i>1$, the next gift is uniform on $\{1,\dots ,i-1\}$ and independent of the previous choices. The remaining fortune is therefore uniform on that same set. At fortune $1$ the process stays there. Consequently the future conditional distribution depends only on the current fortune, proving the Markov property, and the transition matrix on $\{1,\dots ,n\}$ isThe uniform decreasing Markov chain has state $1$ as an absorbing state, and until absorption the fortune strictly decreases, so the hitting time is at most $n-1$.

Let $e_i$ be the expected number of additional transitions to hit $1$, taking $e_1=0$. First-step analysis givesIn particular $e_2=1$. For $i\ge 3$, multiplying this recurrence by $i-1$ and the recurrence for $i-1$ by $i-2$, then subtracting, gives $(i-1)e_i=(i-1)e_{i-1}+1$. Hence $e_i-e_{i-1}=1/(i-1)$ for every $i\ge 2$, and telescoping yieldsThe PDF defines $\tau$ using times $i\ge 1$. If $n=1$ were included, that definition would give $\tau =1$ although the displayed empty sum is zero; the initial instruction to choose an integer between $1$ and $n-1$ also requires $n\ge 2$. The result above uses that intended assumption. A hitting time allowing time zero would have expectation zero from state $1$, but that is a different definition.

The three year states are transient states: from each there is positive probability of entering an absorbing state, and repeated trials make eventual departure certain because $q<1$. The singleton classes $\{L\}$ and $\{G\}$ are closed absorbing states, hence recurrent. Thus the chain has two closed recurrent classes and three transient singleton classes.

At an absorbing state, the remaining absorption time is zero, so $e_0=e_n=0$. Multiplying the linear recurrence relation by three givesThe second finite difference of $k^2$ is two. A particular solution is therefore $-3k^2/2$, while every solution of the corresponding homogeneous linear recurrence relation has the form $A+Bk$. The boundary conditions give $A=0$ and $B=3n/2$, henceThis is the mean absorption time of a lazy symmetric random walk with holding probability $1/3$.

Starting from $X$ at time $n$, the markov property and the time-independent transition rules give $\mathbb{P}(M_{2n}=X\mid M_n=X)=p_n$. ThereforeWrite $p_n=\frac{3}{4}(\frac{3}{4}{)}^n\left(1+\frac{1}{3}(-\frac{1}{3}{)}^n\right)$. The conditional probability becomesThe conditioning selects paths which have avoided the absorbing state up to time $2n$; it is therefore quite different from the unconditioned probability $p_n$, which tends to zero.

This continuous-time multi-state model permits progression $1\to 2$, direct death $1\to 3$, and death after progression $2\to 3$, with no recovery and death an absorbing state. For constant positive rates $a,b,c$, write $\lambda =a+b$. Integrating over the time of the progression arrow givesUse $at{e}^{-ct}$ when $\lambda =c$. The other transition probabilities are $p_{11}=e^{-\lambda t}$, $p_{13}=1-p_{11}-p_{12}$, $p_{22}=e^{-ct}$, $p_{23}=1-e^{-ct}$ and $p_{33}=1$. Unlike a purely sequential irreversible three-state disease model, the direct-death rate contributes to the first-state exit rate. A mixed panel and exact-death likelihood accounts for panel visits and exact death times with different kinds of factors.

A sequential continuous-time Markov chain has transitions $1\to 2\to 3$ with rates $\lambda ,\nu$ and an absorbing state $3$. Its probability of ever leaving state $1$ by time $t$ is $1-e^{-\lambda t}$; its probability of occupying state $2$ is $\lambda (e^{-\lambda t}-e^{-\nu t})/(\nu -\lambda )$ for unequal rates.

For a finite-state continuous-time Markov chain, a transition intensity matrix has nonnegative off-diagonal rates $q_{rs}$ and $q_{rr}=-{\sum }_{s\ne r}{q}_{rs}$. Every row sums to zero. In the time-homogeneous case its transition probability matrix is the matrix exponential $P(t)=e^{tQ}$. An absorbing state has a zero row.

On $\{1,\dots ,n\}$, make $1$ an absorbing state and let each state $i>1$ move uniformly to $\{1,\dots ,i-1\}$. Its expected hitting time of $1$, allowing zero time when started at $1$, is $e_i=H_{i-1}$. First-step analysis gives $e_i=1+(i-1{)}^{-1}{\sum }_{j<i}{e}_j$; subtracting consecutive recurrences yields $e_i-e_{i-1}=1/(i-1)$ and hence the formula.

At each site, a rate-one clock rings and the site copies the state of a randomly chosen neighbor. The model is a Markov process on Ising spin configurations, interpreted through its local update infinitesimal generator. All constant configurations are absorbing state. Tracing copying arrows backwards gives the voter-model duality with coalescing random walks.