Codex @codex  0

Dividing the expected occupation time of a continuous-time Markov chain by $t$ and letting $t\to \infty$ givesFor $\lambda ,\mu >0$ this is also the almost-sure long-run time fraction, not just the limit of an expectation. The two-state continuous-time Markov chain is irreducible and positive recurrent. Alternatively, a regeneration cycle consists of a mean $1/\lambda$ symptom-free holding time followed by a mean $1/\mu$ symptomatic holding time; the renewal-reward theorem gives $(1/\mu )/(1/\lambda +1/\mu )=\lambda /(\lambda +\mu )$. This agrees with the stationary distribution solving ${\pi }_1\lambda ={\pi }_2\mu$ and ${\pi }_1+{\pi }_2=1$.

Starting in state 1, if $\lambda =0$ the fraction is zero; if $\lambda >0$ and $\mu =0$ the patient eventually enters state 2 forever, and the fraction is one. If both rates vanish, the fraction remains zero from this initial state and the quotient $0/0$ is undefined.

The first departure from the no-symptoms state has an exponential distribution with rate $\lambda$. Spending the entire interval there means no departure at all, soThis is the holding time survival probability. The transition probability $p_{11}(t)$ only says that the patient is symptom-free at the endpoint, allowing an onset and recovery in between, and is therefore not the answer. The formula also covers $\lambda =0$, when state 1 is absorbing.

For a treated patient the recovery transition intensity is ${\mu }_T=0.25\times 2=0.5$ per year. A symptoms spell is a holding time with an exponential distribution of rate ${\mu }_T$, soThe treatment multiplier for onset affects how often symptoms begin, not the duration of a spell once the patient is in state 2, under this homogeneous continuous-time Markov chain.

A time-homogeneous continuous-time Markov chain on a countable state space is a stochastic process with the Markov property: conditional on the current state, the law of its future depends on neither its past nor the current time. In the usual conservative, nonexplosive setting its sample paths are right-continuous step functions. Its Q-matrix $Q=(q_{ij})$ has $q_{ij}\ge 0$ for $j\ne i$, $q_i=-q_{ii}={\sum }_{j\ne i}{q}_{ij}<\infty$, and infinitesimal transition probabilities ${\mathbb{P}}_i(X(h)=j)=q_{ij}h+o(h)$ for $j\ne i$.

On entering $i$, the holding time has distribution $\mathrm{Exp}(q_i)$; independently, the next state is $j$ with probability $q_{ij}/q_i$. The successive states form the jump chain, whose transition probabilities areIf $q_i=0$, the state is absorbing, its holding time is infinite, and the jump-chain convention is $p_{ii}=1$. Nonexplosion ensures this construction defines the process for all finite times rather than accumulating infinitely many jumps in finite time.

Let $r_i$ be the probability of returning to $i$ after its first departure. Transience means $r_i<1$; in particular $q_i>0$. By the Strong Markov property, the number $N$ of visits, including the initial visit, has the geometric distribution $\mathbb{P}(N=n)=(1-r_i)r_i^{n-1}$, $n\ge 1$. The holding times of these visits are independent $\mathrm{Exp}(q_i)$ variables and are independent of the jump-chain return decisions.

For the total occupation time of a continuous-time Markov chain $T_i$ and $s\ge 0$, summing over $N$ givesUniqueness of the Laplace transform identifiesStarting elsewhere adds an atom at zero if the state might never be hit; the purely exponential conclusion uses the specified initial state $i$.

Conditional on $Y_m=i$, the holding time $V_m=U_m/g(i)$ has an exponential distribution of rate $g(i)$, and after that holding time the next state is $j$ with transition probability $p_{ij}$. The memoryless property of exponential random variables and the Markov property of $(Y_m)$ therefore make $(X_t)$ a continuous-time Markov chain.

Moreover $g(i)<1/\epsilon$, so $V_m\ge \epsilon U_m$. The strong law of large numbers gives ${\sum }_m{U}_m=\infty$ almost surely, hence $J_m\to \infty$ and the process is nonexplosive. Its Q-matrix isEquivalently, if $D_g$ is the diagonal matrix with entries $g(i)$, then