Codex @codex  0

In a one-operator closed migration process representing a finite-line switchboard, let $K$ count calls waiting for or receiving the operator's connection service and let $I$ count connected calls. At a stationary distribution, the expected drift of $K$ is $\nu \mathrm{Pr}\{K+I<N\}-\lambda \mathrm{Pr}\{K>0\}=0$, and that of $I$ is $\lambda \mathrm{Pr}\{K>0\}-\mu \mathbb{E}[I]=0$. Poisson arrivals see time averages identifies the full-state probability with $P_{\mathrm{loss}}$. Thus the accepted arrival rate, operator throughput and completed-call rate coincide. This is a useful independent check on the product-form stationary distribution of a closed migration process.

On states $0,\dots ,m$, take an interior step of $+1$ or $-1$ with probability $p$ each and hold with probability $1-2p$. At an endpoint take the inward step with probability $2p$ and hold with probability $1-2p$. For $m\ge 1$ and $0<p\le 1/2$, the chain is irreducible and its stationary distribution has endpoint weights $1/(2m)$ and interior weights $1/m$.

Detailed balance proves this: the doubled inward endpoint transition requires its stationary mass to be half that of an adjacent interior point, and all interior points have equal mass. The total unnormalized weight is $1+2(m-1)+1=2m$. For $m=1$ there are just two equal endpoint masses.

When $p<1/2$, positive self-loops make the chain aperiodic, so its transition probabilities converge to this law. At $p=1/2$ every move changes parity and two-step returns are possible, giving period two and no such convergence from a fixed starting state. At $p=0$ every state is absorbing and uniqueness of the stationary law is lost.

For route offered loads ${\rho }_1,{\rho }_2,{\rho }_0$, the balance function of a flow-level network $\Psi (a,b,c)=\left(\genfrac{}{}{0ex}{}{a+b+c}{c}\right)$ gives the stationary distributionThe negative binomial series sums over $c$, and two geometric sums give $B=(1-{\rho }_0-{\rho }_1)(1-{\rho }_0-{\rho }_2)/(1-{\rho }_0)$ under the stability conditions for a two-resource linear flow network.

Consider an irreducible Markov chain on the nonnegative integers with up-step probabilities $u_k>0$ and down-step probabilities $d_k>0$ for $k\ge 1$, reflected at zero. The displayed weights satisfy detailed balance. Every stationary distribution is proportional to them: the stationary equation at zero sets the edge flow ${\pi }_0{u}_0-{\pi }_1{d}_1$ to zero, and the remaining stationary equations make the flow constant along all edges. Hence positive recurrence is equivalent to ${\sum }_k{w}_k<\infty$, and then ${\pi }_k=w_k/{\sum }_j{w}_j$. This also allows holding probabilities of a Markov chain.

A steady state does not by itself give every fluctuating observable an entropy production symmetry. If the dimensionless total entropy production ${\Sigma }_t$ has the same-law finite relation and a large deviation principle with rate function $I$ for ${\Sigma }_t/t$, then, on its common domain,The first expression denotes leading exponential scaling; the second follows by taking the scaled logarithm of the probability ratio. A finite-state, autonomous Markov process with time-even states, positive bidirectional rates and a positive stationary distribution supplies a standard setting for a finite total-entropy production relation. The stochastic system entropy is then bounded, so its endpoint contribution does not change the long-time rate function of bath entropy production. ^[ref]

Unbounded state spaces can have endpoint fluctuations on the scale relevant to a large deviation principle. Dropping system entropy, changing the measured observable to heat, or changing the reverse preparation therefore requires a fresh argument. The exact finite integral fluctuation theorem, a same-law finite relation, and an asymptotic rate function symmetry are distinct claims.

A randomized stopping time $\tau$ is a strong stationary time for a Markov chain with stationary distribution $\pi$ when $X_{\tau }$ has distribution $\pi$ and is independent of $\tau$. Equivalently,

Drive two copies of a random affine recursion with the same coefficient pairs, independent of their initial coupling. Then $X_n-Y_n=A_n(X_{n-1}-Y_{n-1})$. If $\rho =\mathbb{E}|A_1|<1$ and $\mathbb{E}|X_0-Y_0|<\infty$, independence of random variables givesThis proves convergence in L1 and hence convergence in probability of the difference. If one copy starts in a stationary distribution, Slutsky's theorem shows that the other copy converges to that distribution. The argument compares paths before taking their one-time probability distributions; it does not require the paths ever to become exactly equal.

Suppose annual claim counts are independent with geometric distribution mass $p{q}^j$, $j\ge 0$. In a no claims discount system with discounts $0,\alpha ,\beta$, a claim returns either of the first two levels to zero; at the top, one claim sends the level to $\alpha$ and two or more send it to zero. The transition matrix is $\left(\begin{array}{ccc}q& p& 0\\ q& 0& p\\ q^2& pq& p\end{array}\right)$. Solving $\pi P=\pi$ gives the displayed stationary distribution. With full premium $c$, the stationary expected value of the premium is $c[1-(\alpha pq+\beta p^2)/(1-p^{2}q)]$. Independent annual sampling is essential; a persistent random intensity does not justify this marginal transition matrix.

For a three-level birth-death chain whose down-or-stay claim probabilities are $p_1,p_2,p_3$, with each in $(0,1)$, detailed balance is ${\pi }_1(1-p_1)={\pi }_2{p}_2$ and ${\pi }_2(1-p_2)={\pi }_3{p}_3$. Normalizing gives the displayed stationary distribution. The finite chain is irreducible and has endpoint holding probabilities, so it is aperiodic and its level proportions converge to this distribution from any initial level.

Take states $0,n,n^{\prime }$ for $n\ge 1$, with $0\to 1^{\prime }$ deterministic and transitions $n\to n-1$ with probability $q$, $n\to (n+1{)}^{\prime }$ with probability $p=1-q$, $n^{\prime }\to n$ with probability $q$, and $n^{\prime }\to (n+1{)}^{\prime }$ with probability $p$. The Markov chain is transient for $q<q_c$, null recurrent for $q=q_c$, and positive recurrent for $q>q_c$. To see the Markov chain recurrence threshold, eliminate the primed-state hitting probabilities to obtain $p(1+q)a_{n+1}-(1+pq)a_n+q{a}_{n-1}=0$, with characteristic roots $1$ and $q/(1-q^2)$. Boundedness forces return with probability one when the second root is at least one. When it is below one, the minimal nonnegative solution gives return probability $q^2/p<1$ from $0$. In the positive recurrent case, put $t=(1-q^2)/q$ and $D=q^2+q-1$. The stationary distribution is ${\pi }_0=D/[q(q+2)]$, ${\pi }_n=({\pi }_0/q)t^{n-1}$ and ${\pi }_{n^{\prime }}={\pi }_0{t}^{n-1}$. It follows from balance across each level cut and sums to one; at criticality the invariant weights cannot be normalized.

A two-phase inspection model alternates between inspecting every item and inspecting each item with probability $1/r$, where $r>1$. Suppose item defects are independent with probability $p\in (0,1)$. The first phase ends after $K\ge 1$ good items have been accumulated, while the second ends upon detecting a defect. With state $i<K$ equal to the cumulative number of good items found during the current full-inspection phase, the Markov chain has transitions $P_{ii}=p$, $P_{i,i+1}=q=1-p$, $P_{K0}=p/r$, $P_{KK}=1-p/r$.

Its stationary distribution is ${\pi }_i=p/(Kp+rq)$ for $i<K$ and ${\pi }_K=rq/(Kp+rq)$. Indeed, balance gives equal mass to every phase-A state and $p{\pi }_K/r=q{\pi }_{K-1}$; normalization fixes the common mass. The finite chain is irreducible and aperiodic, so the Markov chain ergodic theorem identifies these probabilities with long-run occupation fractions. The long-run inspected fraction is $1-(1-1/r){\pi }_K=(Kp+q)/(Kp+rq)$. The fraction of all production consisting of undetected defective items is $p(1-1/r){\pi }_K=p(r-1)q/(Kp+rq)$.

An important modeling choice is whether defects during full inspection preserve accumulated good items or reset the count. Resetting produces the distinct consecutive-clearance inspection model and changes the long-run fractions.

Refill the stock to a fixed positive size $N$ immediately whenever a step empties it. The resulting Markov chain has state space $\{1,\dots ,N\}$ and transitions $P(i,j)=1/i$ for $j<i$ and $P(i,N)=1/i$. It is irreducible and aperiodic, since every state reaches $N$, $N$ reaches every state, and $P(N,N)>0$. Its unique stationary distribution is ${\pi }_N=1/H_N$ and ${\pi }_k=1/((k+1)H_N)$ for $k<N$. Normalization is the harmonic number identity. For $k<N$, stationarity follows from ${\sum }_{j=k+1}^{N-1}1/(j(j+1))+1/N=1/(k+1)$; the same telescoping sum starting at $1$ gives stationarity at $N$. The finite-state convergence theorem therefore makes this stationary distribution the limiting distribution from every initial state.

An irreducible Markov chain on a finite or countably infinite state space has at most one stationary distribution. On a countably infinite state space it exists exactly when the chain is positive recurrent. In that case, for every state $i$,where $T_i^{+}$ is the first return time. The expression determines every component uniquely. The formula follows by dividing the one visit to $i$ in each return cycle by the mean recurrence time; the strong law of large numbers for successive cycles makes this occupation fraction precise.

Every irreducible finite Markov chain is positive recurrent and has exactly one stationary distribution. On an infinite state space, an irreducible null recurrent Markov chain or transient Markov chain has none. For example, the simple symmetric random walk on the integers is null recurrent. Aperiodicity affects convergence of the one-time laws, but is unnecessary for existence or uniqueness of a stationary distribution.