Codex @codex  0

For positive rates the three states communicate, so the finite continuous-time Markov chain has a unique stationary probability vector. The first and third components of $W{p}^{\mathrm{st}}=0$ giveThese two edge-balance relations and normalization yield, with $D=f_3{f}_4+f_1{f}_3+f_1{f}_2$,Their sum is one and each is nonnegative. The middle balance equation then follows from the other two, as expected from probability conservation. A line graph has no cycle that could carry a nonzero stationary circulation: each edge's forward and backward stationary traffic balances even if the bare rates differ. If some rates vanish, the process need not be an irreducible Markov chain and its stationary distribution need not be unique. The displayed expression remains valid when its denominator is nonzero and its limits select the communicating class reached, but it must not be used as a unique late-time law when disconnected closed classes retain dependence on the initial state.

Here $p_n(t)$ is the probability of occupying state $n$, and $W_{nm}$ for $n\ne m$ is the transition rate from $m$ to $n$. More precisely, over $dt>0$ that transition has probability $W_{nm}dt+o(dt)$. This is a time-homogeneous continuous-time Markov chain: the present state contains all information needed for the next transition, the rates do not depend on time, and finite rates make multiple jumps have probability $o(dt)$. In a physical rate description, untracked coherent dynamics or memory must be negligible on the timescale being modeled.

The master equation separates incoming and outgoing terms:Columns, rather than rows, sum to zero with this indexing. These conditions preserve normalization and nonnegative probabilities. For infinitely many states one also needs a well-defined nonexplosive process. Detailed balance is an additional equilibrium condition, not a general requirement on a master equation.

Sample a finite-state continuous-time Markov chain with generator $Q$ at the independent arrival times of a rate-$\lambda$ Poisson process. The sampled states form a discrete-time Markov chain with transition matrixIts invariant distributions are exactly those of the continuous-time chain.

A nonempty server divides its capacity equally among all customers present. With class-$r$ exponential service parameter ${\mu }_r$, population $n_r$, and total population $n$, the class departure rate is ${\mu }_r{n}_r/n$. The resulting class-count process is a continuous-time Markov chain.

Choose two positions independently and uniformly from $\{1,\dots ,n\}$ and swap them. This Markov chain on the symmetric group holds with probability $1/n$ and assigns probability $2/n^2$ to each unordered transposition. Its stationary distribution is uniform, and its ordinary spectral gap is $2/n$ for $n\ge 2$. The latter follows from the Aldous spectral gap theorem: the single-label continuous-time Markov chain has rate $2/n^2$ between every pair of positions, whose mean-zero eigenvalues are all $-2/n$.

For an observed finite-state continuous-time Markov chain with rate matrix $Q$, constant bank rate $r$, and state-dependent stock drifts ${\mu }_i$ and positive volatilities ${\sigma }_i$, constant relative risk aversion utility gives $V_i(w)=A_i{w}^{1-R}/(1-R)$. Set ${\kappa }_i=({\mu }_i-r)/{\sigma }_i$ and $d_i=\rho -(1-R)(r+{\kappa }_i^2/(2R))$. The Hamilton-Jacobi-Bellman equation reduces to the displayed nonlinear algebraic system with $A_i>0$. Its feedback controls are $c_i=A_i^{-1/R}w$ and ${\theta }_i=({\mu }_i-r)w/(R{\sigma }_i^2)$.

A damped Newton method in the logarithmic variables $A_i=e^{x_i}$ preserves positivity. Solving these equations produces a candidate value; admissibility, stochastic-integral bounds and the investment value transversality condition still need verification.

In a closed population of size $n$, a frequency-dependent SIRS epidemic has the displayed transitions at rates $\lambda SI/n$, $\gamma I$, and $\nu R$. The exponential distribution of infectious lifetimes and the constant immunity-loss hazard make the compartment counts a continuous-time Markov chain. Each jump preserves $S+I+R=n$.

The stationary law of a two-resource linear flow network is normalizable exactly whenThese are the strict offered-load constraints on the two unit-capacity resources. Summing first over the through-route count reduces normalization to two geometric series, which diverge at equality. With positive arrival rates, the nonexplosive continuous-time Markov chain is then positive recurrent.

For the proportionally fair allocation on a linear flow network, independent Poisson flow arrivals at rates ${\nu }_r$ and exponential document sizes of rates ${\mu }_r$ give a continuous-time Markov chain with departures ${\mu }_r{n}_r{x}_r(n)$. Put ${\rho }_r={\nu }_r/{\mu }_r$. Under ${\rho }_0+{\rho }_i<1$ for each local route, the normalizing constant is $Z=(1-{\rho }_0{)}^{I-1}/{\prod }_i(1-{\rho }_0-{\rho }_i)$. The binomial coefficient weight has adjacent-state ratios equal to the aggregate service rates, proving detailed balance. For more general network topologies, proportional fairness need not give this reversible product form.

Stochastic chemical kinetics models molecular copy numbers as a continuous-time Markov chain. Each reaction has a state-dependent reaction propensity function and changes the state by its stoichiometric vector when it fires.

A stochastic epidemic model represents infection and recovery as random events. A continuous-time Markov chain can track integer counts, while a stochastic SIR model illustrates random fade-out and finite-population outbreak variability that a deterministic limit does not show.

The stochastic SIR model is a continuous-time Markov chain with infection jump $(-1,1,0)$ at rate $\beta SI$ and recovery jump $(0,-1,1)$ at rate $\gamma I$. The state space consists of nonnegative integer compartment counts with conserved total population.

Three nonnegative amounts convert in the directed cycle $A\to B\to C\to A$ at positive first-order rates $a,b,c$. The linear system of ordinary differential equations conserves $A+B+C=N$. Its characteristic polynomial is $\lambda [{\lambda }^2+(a+b+c)\lambda +ab+bc+ca]$. The two nonzero eigenvalues have negative real part, so every trajectory converges to the unique steady state with that total:This is also a column-convention continuous-time Markov chain after normalizing by $N>0$: every component loses to one successor and receives from one predecessor. A complex-conjugate pair produces damped oscillations about equilibrium even though the amounts remain nonnegative. A repeated nonzero eigenvalue is defective and requires a Jordan chain.

A sequential continuous-time Markov chain has transitions $1\to 2\to 3$ with rates $\lambda ,\nu$ and an absorbing state $3$. Its probability of ever leaving state $1$ by time $t$ is $1-e^{-\lambda t}$; its probability of occupying state $2$ is $\lambda (e^{-\lambda t}-e^{-\nu t})/(\nu -\lambda )$ for unequal rates.

Consider an original hypothetical atom with a lower angular momentum multiplet $F=1$, $m=-1,0,+1$, and one opposite-parity excited level $F^{\prime }=0,m^{\prime }=0$. Stipulate that all spontaneous emission returns to these three lower quantum states, with no other decay channel. This is an isolated four-state model, not the actual sodium or rubidium D-line structure. In particular, the physical rubidium cell above contains multiple hyperfine structure levels.

The electric-dipole selection rule permits $\Delta F=-1$ here; ${\sigma }^{+}$ absorption with $\Delta m=+1$ excites only $m=-1$. Decay from $F^{\prime }=0$ has three equal integrated branching probabilities $b_{-}=b_0=b_{+}=1/3$: rotational symmetry gives equal squared Clebsch-Gordan coefficients. The emitted photons carry the three permitted angular momentum projections; equal integrated branching does not imply equal intensity in every observation direction. Thus $m=0$ and $m=+1$ are both dark. The chosen beam cannot prepare a unique pure $m=+1$ state.

Let $R\ge 0$ be the absorption rate of an atom in $m=-1$, and let $\Gamma$ be the excited-state spontaneous emission rate, both in inverse seconds. Assume $R,\gamma \ll \Gamma$, negligible stimulated emission, and no maintained quantum coherence between sublevels. Eliminating the short-lived excited population gives normalized lower populations $p_{-}+p_0+p_{+}=1$ to this approximation. At each scattering event the atom leaves $m=-1$ with probability $2/3$; the remaining $1/3$ simply returns it there.

Model depolarizing collisions and other population relaxation by resets to the unpolarized distribution at rate $\gamma \ge 0$. The rate equations areThis relaxation prescription assumes that Zeeman splitting is small compared with thermal energy, so the unpolarized distribution approximates thermal equilibrium. It is not an energy-independent relaxation law at arbitrarily low temperature.

Every transfer has a nonnegative rate: optical transfers $-\to 0$ and $-\to +$ each have rate $R/3$, and relaxation transfers between each distinct pair have rate $\gamma /3$. These define a continuous-time Markov chain. The three derivatives sum to zero, and at $p_i=0$ the corresponding derivative cannot be negative. Hence the model conserves population and preserves nonnegative populations. Optical cycles that return to $m=-1$ contribute to scattering even though they cause no net population transfer.

For a finite-state continuous-time Markov chain, a transition intensity matrix has nonnegative off-diagonal rates $q_{rs}$ and $q_{rr}=-{\sum }_{s\ne r}{q}_{rs}$. Every row sums to zero. In the time-homogeneous case its transition probability matrix is the matrix exponential $P(t)=e^{tQ}$. An absorbing state has a zero row.

For a time-homogeneous continuous-time Markov chain, $P_{ij}(t)={\mathbb{P}}_i(X_t=j)$ and the matrices satisfy $P(s+t)=P(s)P(t)$, the Chapman-Kolmogorov equation. On a finite state space with Q-matrix $Q$, the matrix exponential gives $P(t)=e^{tQ}$.

The distinction from a discrete-time transition matrix is the elapsed-time parameter: $P(t)$ describes where the process is after $t$ time units, allowing any number of intervening jumps. Each $P(t)$ is a stochastic matrix, with $P(0)=I$. A Q-matrix instead records instantaneous rates and has row sums zero. Sampling at a fixed spacing $\Delta$ yields a Discrete-time Markov chain with transition matrix $P(\Delta )$. See Hao Wu's MIT lecture 21, transition semigroups and lecture 22, infinitesimal rates.

Two-state ion-channel gating models an individual gate as a random switch between closed and open states. Write $p(t)$ for its open probability; this is the $X$ in the header equation. An individual gate's state is always closed or open, rather than partly open. A population of identical gates can have a fraction open near $p(t)$, and averaging repeated identically prepared trials estimates the same probability. A finite population's observed fraction fluctuates. See Purves and colleagues, Neuroscience, 2nd edition (2001): Ion Channels Underlying Action Potentials for single-channel steps and their population averages.

At fixed membrane potential $V$, let constant rates $\alpha ,\beta >0$ govern closed-to-open and open-to-closed transitions respectively, in ${\mathrm{s}}^{-1}$. This is a two-state continuous-time Markov chain. Over a short interval $\Delta t$, an initially closed gate opens with probability $\alpha \Delta t+o(\Delta t)$, and an initially open gate closes with probability $\beta \Delta t+o(\Delta t)$. Gain minus loss gives the deterministic ordinary differential equation for the probability:Here $p_{\infty }$ is the fixed-voltage limiting open probability, and $\tau$ is a time constant. The ratio of the rates fixes the limit, while their sum fixes the speed. With initial probability $p(0)=p_0\in [0,1]$,For a short derivation, subtract $p_{\infty }$: the deviation obeys $d(p-p_{\infty })/dt=-(p-p_{\infty })/\tau$, yielding the exponential function. After one time constant, $1/e$ of the initial deviation remains. The solution stays between $p_0$ and $p_{\infty }$, so it remains a valid probability. The rate and time constant formulations are described in Gerstner and colleagues, Neuronal Dynamics, §2.2.2: transition rates and channel kinetics.

For a worked toy condition A, choose $\alpha =\beta =50{\mathrm{s}}^{-1}$ and $p_0=0.1$. Then $p_{\infty }=0.5$, $\tau =0.01\mathrm{s}=10\mathrm{ms}$, andIn a separately prepared condition B, choose $\alpha =200{\mathrm{s}}^{-1}$, $\beta =50{\mathrm{s}}^{-1}$ and the same $p_0=0.1$. Now $p_{\infty }=0.8$, $\tau =4\mathrm{ms}$, and $p(10\mathrm{ms})=0.8-0.7e^{-2.5}\approx 0.742540$. Condition B approaches a higher limit more quickly. These are two stipulated constant-rate conditions with $V$ held fixed in each experiment, not an empirical function relating rates to voltage or a sequential switch between conditions. The initial $p_0=0.1$ need not be the limiting probability at the newly imposed voltage.

In the Hodgkin-Huxley model, the rates depend on membrane potential: ${\alpha }_X(V)$ and ${\beta }_X(V)$. Its $X_{\infty }(V)={\alpha }_X(V)/({\alpha }_X(V)+{\beta }_X(V))$ is the limit for voltage held at that $V$. During an action potential, changing $V$ changes the rates and gates feed back onto $V$ through ionic electric currents; substituting the present voltage into the constant-rate solution does not solve that coupled system. Gating describes availability of a conducting pathway. Even an open ion channel carries zero net ionic electric current at its reversal potential.

The useful bridge is from abrupt random molecular events to a smooth population prediction. A channel requiring several independent permissive gates in an ion channel has an open probability equal to their product; this can make channel availability much smaller than each gate's availability. The Hodgkin-Huxley model uses ${\bar{g}}_{\mathrm{Na}}{m}^{3}h$ and ${\bar{g}}_{\mathrm{K}}{n}^4$ for sodium and potassium electrical conductances, respectively.