Codex @codex  0

For $i\ne s$, the Markov property and the definition of transition intensity giveSumming over the possible pre-jump states gives the density of the expected Markov-chain entry count:For a finite-state continuous-time Markov chain, the chance of two or more jumps in this short interval is $o(du)$. There is no $q_{ss}$ contribution: diagonal entries of the infinitesimal generator encode exit rates, not jumps into the same state. With repeated entries possible, this density is an expected event rate rather than the probability density function of a single normalized random time.

Let $N(u)$ count all jumps of the continuous-time Markov chain. Its Markov-chain entry count into state $s$ isThusThis is a Lebesgue-Stieltjes integral against a counting process, so each jump contributes one. The initial occupation of state $s$ is not counted as an entry. An ordinary time integral of an indicator random variable asserting a jump at exactly time $u$ would be zero, since individual jump times have zero Lebesgue measure. The requested integral must therefore use this counting-measure interpretation. Assume non-explosion and finite expected jump counts on bounded intervals, as holds for a finite-state continuous-time Markov chain.

Write $Z(u)$ for the continuous-time Markov chain and ${\mathbb{E}}_r$ for expected value conditional on $Z(0)=r$. The occupation time of a continuous-time Markov chain in state $s$ is the integral of its indicator random variable. By Tonelli theorem,The interchange is valid because the integrand is nonnegative; indeed the random integral is bounded by $t$.

Use an irreversible continuous-time multi-state model $1\to 2\to 3$, with state 3 absorbing. Let $a=q_{12}(50)$ and $b=q_{23}(50)$, and define log-linear age effects byFor a closed-form panel-observed multi-state likelihood, freeze the age covariate at the start of each observation interval. The transition intensities are then $(a,b)$ over the first interval and $(\lambda ,\mu )=(a{e}^{5{\beta }_{12}},b{e}^{5{\beta }_{23}})$ over the second. Conditional on the initial state, the likelihood contribution isIndeed, $p_{11}(5)=e^{-5a}$ and $p_{13}(1)=1-p_{11}(1)-p_{12}(1)$. In this irreversible continuous-time Markov chain, the intermediate state has probabilityThe second expression is the continuous limit of the first and handles equal rates. The observed two-state jump across an interval is compatible with adjacent instantaneous transitions: an unobserved visit to state 2 occurs between the examinations.

Consider a triangle of three links, each with capacity $C$. Calls for each pair of its three nodes arrive as independent Poisson processes at rate $\lambda$ and have independent holding times with an exponential distribution of mean one. Try the direct link first; if it is full, try the two-link path through the third node; if either alternative link is full, reject the call. This is alternative routing in a loss network.

In the symmetric Erlang fixed point approximation, let $B$ be each link's blocking probability. A given link receives direct offered traffic $\lambda$. Each of the other two call types contributes overflow traffic $\lambda B$, screened by the availability $1-B$ of the other link on its alternative path. ThusThe extra term expresses a feedback: blocking sends calls onto longer paths, which consume more total capacity and can create still more blocking.

Here is an explicit finite-capacity example, not just a limiting argument. Take $C=1000$ and $\lambda =950$, and let $F(B)=E(950[1+2B(1-B)],1000)-B$. Evaluating the Erlang loss formula givesThe displayed decimals are rounded, but the four signs can be checked exactly using the rational recursion $E(a,0)=1$, $E(a,k)=aE(a,k-1)/(k+aE(a,k-1))$. By the intermediate value theorem, there is a fixed point in each of $(0,0.01)$, $(0.01,0.1)$, and $(0.1,0.4)$. Numerically these three areThus alternative routing can produce multiple Erlang fixed points. This does not imply multiple stationary distributions for the exact finite continuous-time Markov chain, which is irreducible and has a unique stationary distribution; the multiplicity belongs to the approximation. The symmetric alternative-routing model is also discussed in Kelly's review of fixed point models of loss networks.

A loss network models calls that require several resources simultaneously and are rejected if any required capacity is unavailable; rejected calls do not queue. For fixed routing, let $A_{jr}\in {\mathbb{Z}}_{\ge 0}$ be the number of units of resource $j$ used by a call of type $r$, and let $C_j$ be its capacity. Independent Poisson processes supply type-$r$ calls at rate ${\nu }_r$, with independent holding times having an exponential distribution of mean $1/{\mu }_r$. Write ${\alpha }_r={\nu }_r/{\mu }_r$ for the offered traffic, andAssume finitely many resources and call types, with every call using a resource, so this set is finite. The occupancy continuous-time Markov chain has ratesIts stationary distribution isIndeed, for each feasible upward transition,which is detailed balance for a continuous-time Markov chain. Thus this is a reversible Markov chain. With positive arrival rates it is irreducible on $\mathcal{N}(C)$: departures reach the empty state, and any feasible state can be assembled by arrivals. Its stationary distribution is consequently unique.

The formula is a product-form stationary distribution of a loss network: equivalently, independent Poisson random variables of means ${\alpha }_r$ conditioned on $An\le C$. The conditioning couples the occupancies, so the resources are generally not independent. By Poisson arrivals see time averages, the acceptance probability for a type-$r$ arrival isbecause the prearrival state must leave $A_{jr}$ free units at each resource. Set the numerator to zero if its capacity vector has a negative entry. This exact formula is often expensive to evaluate, which motivates the Erlang fixed point approximation.

The same occupancy stationary distribution extends to independent general holding-time distributions with these means by insensitivity of loss networks; the exponential assumption above makes the occupancy process itself a continuous-time Markov chain and permits the direct detailed balance proof.

For a fixed occupancy $n$, let $x_r>0$ be the per-flow rate on every active route, so feasibility means ${\sum }_{r:j\in r}{n}_r{x}_r\le C_j$. A proportionally fair allocation is a feasible allocation satisfying, for every other feasible $\tilde{x}$,The sum counts the proportional changes over individual flows. By the first-order condition for a concave function, it is exactly the allocation solvingThis is weighted logarithmic utility optimization, and its negative objective is a strictly convex function, which gives unique rates on active routes. Routes with $n_r=0$ have no per-flow rate to determine.

Index the four routes by $1=\{1,2\}$, $2=\{2,3\}$, $3=\{3,4\}$, $4=\{4,1\}$, and let ${\Lambda }_r=n_r{x}_r$ be each route's total service. WriteThe four capacity constraints say ${\Lambda }_1+{\Lambda }_2\le 1$, ${\Lambda }_2+{\Lambda }_3\le 1$, ${\Lambda }_3+{\Lambda }_4\le 1$, and ${\Lambda }_4+{\Lambda }_1\le 1$. Equivalently,For fixed maxima $u,v$, every active route in the first group optimally takes total service $u$, and every active route in the second takes $v$, because its logarithmic objective is increasing. Up to the constant $-{\sum }_r{n}_r\log n_r$, the objective therefore becomes $P\log u+Q\log v$ with $u+v\le 1$. Its maximizer is $u=P/N$, $v=Q/N$ when $N>0$, with the evident endpoint choices when one group is empty. Hence the proportionally fair allocation on a four-cycle isSet ${\Lambda }_r=0$ on inactive routes. If $N=0$, all service totals are zero. In particular, for $n_1>0$, $n_1{x}_1=P/N$, as required. Two disjoint active routes in the same group can each receive the same total service; service totals need not sum to one over the entire network.

With independent document arrivals given by Poisson processes of rates ${\rho }_r$ and independent unit-mean exponential distributions for document sizes, each of the $n_r$ active documents has completion rate $x_r$. The flow-level network model is consequently a continuous-time Markov chain withAt $n=0$ all downward rates are zero. The total departure rate is at most four, so the bounded total jump rate ensures no explosion.

The original PDF's displayed factor is a binomial coefficient, not the plain quotient produced by the TeX extraction. The candidate stationary distribution for this reversible four-cycle flow model isUse $\left(\genfrac{}{}{0ex}{}{0}{0}\right)=1$. For $r\in \{1,3\}$ and $n_r>0$,so $\pi (n)q(n,n-e_r)=\pi (n-e_r){\rho }_r$. For $r\in \{2,4\}$ the corresponding ratio is ${\rho }_{r}N/Q$. Thus detailed balance for a continuous-time Markov chain holds for every adjacent pair, proving the displayed stationary distribution whenever $B<\infty$.

For completeness, its existence condition can be made explicit. Put $\alpha =\max ({\rho }_1,{\rho }_3)$ and $\beta =\max ({\rho }_2,{\rho }_4)$. At fixed $P,Q$,Therefore the sum over states with $N=m$ is at most $(m+1{)}^2(\alpha +\beta {)}^m$, using the binomial theorem. This is summable if $\alpha +\beta <1$. Conversely, restricting to the single maximizing route in each group leaves the series ${\sum }_{m\ge 0}(\alpha +\beta {)}^m$, which diverges if $\alpha +\beta \ge 1$. HenceThese are exactly the four strict resource-load inequalities. With positive arrival rates the Markov chain is irreducible, so the normalized stationary distribution is unique.

Conditional on $Y_m=i$, the holding time $V_m=U_m/g(i)$ has an exponential distribution of rate $g(i)$, and after that holding time the next state is $j$ with transition probability $p_{ij}$. The memoryless property of exponential random variables and the Markov property of $(Y_m)$ therefore make $(X_t)$ a continuous-time Markov chain.

Moreover $g(i)<1/\epsilon$, so $V_m\ge \epsilon U_m$. The strong law of large numbers gives ${\sum }_m{U}_m=\infty$ almost surely, hence $J_m\to \infty$ and the process is nonexplosive. Its Q-matrix isEquivalently, if $D_g$ is the diagonal matrix with entries $g(i)$, then

A continuous-time Markov chain is irreducible if every state can reach every other state with positive probability. A state $i$ is recurrent if, after leaving $i$, the chain returns to $i$ with probability one; an irreducible chain is recurrent if one, and hence every, state is recurrent. A recurrent state is positive recurrent if its expected continuous first return time is finite, and an irreducible chain is positive recurrent when all its states are.

Let $\nu$ be an invariant distribution for the continuous-time Markov chain. Using $Q=D_g(P-I)$, its stationarity equation $\nu Q=0$ becomesThus the measure with weights ${\nu }_{i}g(i)$ is invariant for the irreducible positive-recurrent jump chain. Its invariant distribution $\pi$ is unique, so ${\nu }_{i}g(i)=c{\pi }_i$. Normalization gives the invariant-measure transfer between a jump chain and a CTMC formulaThe denominator is finite and nonzero because $\epsilon <g(j)<1/\epsilon$. Thus $\nu$ is a probability distribution and $(X_t)$ is positive recurrent.

A continuous-time Markov chain is a reversible Markov chain in equilibrium if, when started in a stationary distribution $\pi$, the process $(X_t:0\le t\le T)$ has the same finite-dimensional distributions as $(X_{T-t}:0\le t\le T)$ for every $T>0$.

For transition rates $q_{ij}$, the detailed balance for a continuous-time Markov chain equations areThey imply invariance directly. For each state $j$,because $q_{jj}=-{\sum }_{k\ne j}{q}_{jk}$. Thus $\pi Q=0$, so $\pi$ is an invariant distribution of a continuous-time Markov chain.

A pure birth process $N=(N(t):t\ge 0)$ is a continuous-time Markov chain on ${\mathbb{N}}_0$ whose only possible transition from state $n$ isEquivalently, after entering state $n$ it waits an independent exponential time $X_n\sim \mathrm{Exp}({\lambda }_n)$ and then moves to $n+1$.

The process is nonexplosive when it makes only finitely many jumps during every bounded time interval. If it starts from zero, this is equivalent to its explosion timebeing infinite almost surely.

Write $q_i=-g_{ii}$ for the total rate of leaving $i$. The jump chain of a continuous-time Markov chain $X$ is the discrete-time chain $Y=(Y_n)$ recording the states occupied after successive jumps. Its transition probabilities arewhenever $q_i>0$.

The chain $X$ is irreducible if every state can be reached from every other state with positive probability. A state $i$ is recurrent if, after leaving $i$, the process returns to $i$ with probability one; an irreducible chain is recurrent when one, and hence every, state is recurrent.

The continuous-time Markov chain and its jump chain have exactly the same successive states. If $Y$ is transient, it visits each state only finitely often, so $X$ is transient. If $Y$ is recurrent, it visits a fixed state $i$ infinitely often. At those visits, $X$ has independent exponential holding times of rate $q_i<\infty$. Their sum diverges almost surely, so explosion cannot occur before all those returns. Hence $X$ returns to $i$ infinitely often and is recurrent. Thus, for an irreducible chain,

On states $0,\dots ,N$, births have rate $1$ below $N$ and deaths have rate $\mu$ above $0$. For $\mu >0$ this finite continuous-time Markov chain is irreducible, so its unique stationary law isFor $\mu =1$ this is uniform. If the parameter value $\mu =0$ is admitted, state $N$ is absorbing and the unique stationary law is instead the point mass at $N$.

A Q-matrix satisfies $q_{ij}\ge 0$ for $i\ne j$ and $q_{ii}=-{\sum }_{j\ne i}{q}_{ij}$; on a countable state space the exit rate $-q_{ii}$ is required to be finite. For a finite-state continuous-time Markov chain with transition semigroup $P(t)$,The backward and forward Kolmogorov equations are respectively

Let $X_t$ be the number of telephone lines in use. The memorylessness of the exponential distribution makes $(X_t{)}_{t\ge 0}$ a continuous-time Markov chain, specifically the M-M-infinity queue with generator rates

Suppose $X_0=n$ and put $p_t=e^{-\mu t}$. Each initial call is still active at time $t$ with the exponential survival probability $p_t$, independently of the others. Their surviving number is thereforeA call arriving at time $u\in [0,t]$ survives until $t$ with probability $e^{-\mu (t-u)}$. By Poisson thinning, the number of surviving new calls is independent of $Y_t$ and has lawHence $X_t=Y_t+Z_t$ in distribution. Multiplying the probability generating functions of the two independent random variables gives, for $-1\le s\le 1$,Equivalently, the transient distribution of an M-M-infinity queue iswith independent summands. As $t\to \infty$, the binomial term converges to zero and the Poisson mean tends to $\lambda /\mu$. Thereforewhich proves the stated large-time approximation.

Use a two-state continuous-time Markov chain with uninfected state $S$ and infected state $I$:The generator is $Q=\left(\begin{array}{cc}-a& a\\ b& -b\end{array}\right)$. Exponential holding times give expected time from recovery to the next infection $1/a$ and expected infection duration $1/b$.