Codex @codex  0

Alternative routing lets a call try another resource path when its preferred route is blocked. Overflow onto longer paths can create feedback and multiple solutions of the Erlang fixed point approximation, even though an exact finite irreducible occupancy Markov chain has a unique stationary distribution.

Take $n$ independent steps, each $+1$ with probability $p>1/2$ and $-1$ with probability $q=1-p>0$. Reverse a step sequence by reading it backward and changing each sign; use the same biased law for both experiments. With $K$ positive steps, the forward sequence weight is $p^K{q}^{n-K}$ and its reversed weight is $p^{n-K}{q}^K$. Their logarithm ratio isHere $S_n$ is dimensionless entropy production, not an entropy in joules per kelvin. A Markov chain on a three-state ring gives a state-space realization: start uniformly, take a clockwise step with probability $p$ and a counterclockwise step with probability $q$. The uniform stationary distribution makes endpoint weights cancel. A physical interpretation as exchanged heat requires local detailed balance with bath entropy $\pm k_{B}a$ per directed step; the abstract model alone specifies path probabilities.

For an allowed value $s=a(2k-n)$, the binomial distribution givesThis is an exact same-law finite fluctuation theorem, with no long-time approximation. For $p=3/4$, $n=9$ and $k=6$, $s=3\ln 3$ and $p_n(s)/p_n(-s)=27$. The positive mean is $⟨S_n⟩=na(p-q)$, although the all-negative path has nonzero probability $q^n$. Directly, $⟨e^{-S_n}⟩=(p{e}^{-a}+q{e}^a{)}^n=(q+p{)}^n=1$.

The left panel shows exact probability masses for $p=3/4$, with negative values highlighted; the right panel tests the logarithm ratio against $s$. These are model values, without experimental measurements or simulated measurement noise.

Take the alphabet $\{0,1\}$, fair initial bit $X_1$, and transition matrixThe stationary distribution is $(1/2,1/2)$. Every emitted symbol is a fair bit, while the next symbol flips the previous one with probability $q$. For $q<1/2$, repeating the previous symbol is the better one-step prediction; for $q>1/2$, predicting its opposite is better. The resulting minimum next-symbol error probability is $\min (q,1-q)$, compared with $1/2$ when no history is used.

Given either state, the next-symbol information entropy is $h_2(q)$. ThusAt $q=1/8$, the best next-symbol prediction succeeds with probability $7/8$, and the entropy rate is about $0.54356444$ bits per symbol. The adjacent symbols share mutual information $I(X_1;X_2)=1-h_2(q)\simeq 0.45643556$ bits. This is useful structure even though the single-symbol counts reveal no bias.

At $q=0$, the random initial bit is repeated forever; at $q=1$, it alternates forever. Both have $H(X_1,\dots ,X_n)=1$ and zero entropy rate. At $q=1/2$, the symbols are independent and identically distributed fair bits, giving entropy rate one. Zero rate therefore need not mean that the initial choice was deterministic.

Here $0<p<1$, $\theta >0$, and the upward step is either a birth or immigration, the latter having conditional probability $\theta /(k+\theta )$. At zero immigration occurs with probability one. Put $q=1-p$, $a=\theta /p$ and $r=p/q$. The stationary weights of a reflected birth-death chain areTheir successive ratio tends to $r$, so they are summable for $p<1/2$. For $p\ge 1/2$, $u_k>1/2>d_{k+1}$ makes the weights strictly increasing, ruling out a stationary distribution. Thus positive recurrence holds exactly for $p<1/2$. In that case the binomial series gives ${\sum }_k{w}_k=2(1-r{)}^{-a-1}$, andEvery step reverses parity and a two-step return has positive probability, so the period of a state in a Markov chain is two. Starting at zero, $\mathbb{P}(X_{2n}=0)\to 2{\pi }_0=(1-r{)}^{a+1}$, whereas $\mathbb{P}(X_{2n+1}=0)=0$.

A Boltzmann machine is a stochastic neural network that learns a probability distribution specified by pairwise energies. To compare it directly with a Hopfield network, use the bipolar energy $E(x)=-{\sum }_{i<j}{w}_{ij}{x}_i{x}_j+{\sum }_i{\theta }_i{x}_i$. Here $\beta >0$ controls the preference for low energy, and the partition function $Z={\sum }_x{e}^{-\beta E(x)}$ normalizes all states. In a computational model, $\beta$ is an algorithmic inverse temperature; interpreting it as $1/(k_{B}T)$ additionally requires physical energy units.

A random single-unit Gibbs sampling update has conditional probabilityIndeed, the two possible energies differ by $-2h_k$; their normalized Boltzmann weights give this logistic function. Random selection of one unit at a time satisfies detailed balance. For finite weights and finite positive $\beta$, all conditional probabilities lie strictly between zero and one, so the finite Markov chain can reach every state and can remain in place. Its unique stationary distribution is $p_{\beta }$. A short run need not yet sample that distribution accurately.

As $\beta \to \infty$, a nonzero field selects its sign, approaching asynchronous Hopfield energy descent. At zero field the stochastic rule remains a fair coin, whereas the deterministic convention above retains the old value. At finite inverse temperature, uphill moves permit exploration beyond one local minimum. Recall and learning are different operations: sampling changes the units at fixed weights; learning changes the weights to fit the training data.

An unpinned uniform height driven by Gaussian white noise is Brownian motion. Its variance grows as ${\sigma }^{2}t$ and it has no stationary distribution on the real height axis for $\sigma >0$. Indeed, a stationary characteristic function would obey $\hat{\mu }(k)=\hat{\mu }(k)e^{-{\sigma }^2{k}^{2}t/2}$, incompatible with continuity at zero. Quotienting out the uniform height can still leave stationary shape fluctuations.

The initial segment of a Markov chain Monte Carlo run discarded to reduce dependence on its starting state. A fixed discarded count is a choice to assess using Markov chain Monte Carlo convergence diagnostics, not a proof that the retained sample has reached its stationary distribution.

In the consecutive-clearance inspection model, a run of $K$ consecutive good items is required to leave full inspection. With $q=1-p$, $0<p<1$, its Markov chain has transitions $P_{i0}=p$, $P_{i,i+1}=q$ for $i<K$, and $P_{K0}=p/r$, $P_{KK}=1-p/r$. Thus every defect in the clearance phase resets the count, rather than preserving the cumulative total.

Writing $D=1+(r-1)q^K$, the stationary distribution isTo prove this, balance at $i=1,\dots ,K-1$ gives ${\pi }_i=q^i{\pi }_0$, while balance at $K$ gives $(p/r){\pi }_K=q^K{\pi }_0$. Summing the geometric series and normalizing gives ${\pi }_0=p/D$; balance at zero follows as well. The inspected fraction is $1-(1-1/r){\pi }_K=1/D$. The fraction of all production that is both defective and uninspected is $p(r-1)q^K/D$; the fraction among defective items alone is $(r-1)q^K/D$. All these long-run limits follow from the Markov chain ergodic theorem and the independent inspection coins. The distinction from cumulative clearance matters already at $K=2$.

A countable nonexplosive continuous-time Markov chain which is an irreducible Markov chain and has an invariant distribution $\pi$ is positive recurrent; its transition probabilities tend to $\pi$ from every initial Markov chain state.

To see why continuous time removes periodicity, fix $h>0$. Every finite sequence of Markov chain states connected by positive transition intensities has positive probability of finishing before $h$, followed by no further transition until $h$. Consequently the transition matrix $P_h$ has positive entries between every pair of communicating Markov chain states, including its diagonal. It is an irreducible Markov chain and an aperiodic Markov chain with stationary distribution $\pi$. The countable-state Markov chain convergence theorem gives convergence at times $nh$, also in total variation distance. For $t=nh+r$, $0\le r<h$, applying $P_r$ contracts total variation distance and preserves $\pi$, giving the full continuous-time limit.

For an irreducible Markov chain on a finite or countable state space, a positive recurrent state implies a unique stationary distribution $\pi$. If the chain is also an aperiodic Markov chain, its transition probabilities satisfy the displayed limit for every pair of states. The same limit holds from any initial probability distribution by bounded convergence. Positive recurrence cannot be omitted in the countably infinite case, and aperiodicity is required for ordinary, rather than averaged, convergence. This is the standard discrete-time convergence theorem; the finite-state statement alone does not cover unbounded renewal countdowns.

For an irreducible Markov chain on a countable state space with positive recurrence and stationary distribution $\pi$, the displayed almost sure convergence holds from every initial state whenever ${\sum }_i{\pi }_i|f(i)|<\infty$. In particular, the empirical fraction of visits to $i$ converges to ${\pi }_i$. No aperiodicity is needed. Regeneration at successive returns to a state reduces the result to the strong law of large numbers for independent excursion lengths and rewards. This extends the finite-state Markov chain ergodic theorem.

Detailed balance does not imply that a Markov chain is an irreducible Markov chain: the identity transition matrix is reversible for every probability distribution. Even for an irreducible Markov chain with aperiodicity, the two-state matrix with diagonal $1-\epsilon$ and off-diagonal $\epsilon$, $0<\epsilon <1/2$, has uniform stationary distribution and nontrivial eigenvalue $1-2\epsilon$. From either fixed state its total variation distance after $n$ steps is $\frac{1}{2}(1-2\epsilon {)}^n$, so convergence can be arbitrarily slow. Invariance, reachability and mixing time are distinct questions.

A time-homogeneous Discrete-time Markov chain is a sequence of random variables $Y_0,Y_1,\dots$ indexed by nonnegative integers, with the displayed conditional probabilities. Here ${\mathcal{F}}_n$ is its natural filtration and $P$ is a row-stochastic transition matrix. The Markov property removes dependence on earlier observations once $Y_n$ is specified; the time-homogeneous Markov property makes $P$ independent of $n$.

Consider states $S$ (sunny) and $R$ (rainy), with the constructed transition matrixRows give today's state and columns tomorrow's, in order $S,R$. These numbers define an illustrative model, with one step interpreted as one day; they are not estimates of real weather. Let $q_n=\mathbb{P}(Y_n=S)$. The law of total probability gives the recurrence relationSubtracting $2/3$ makes the deviation multiply by $0.4$ at each step. ThusStarting sunny gives $q_0=1$, $q_1=0.8$, $q_2=0.72$ and $q_3=0.688$; starting rainy gives $q_0=0$ and $q_2=0.56$. The two predictions approach the same stationary distribution while retaining a shrinking trace of the initial probability distribution.

An expected hitting time answers a different question. Starting sunny, let $T_R=\inf \{n\ge 0:Y_n=R\}$. Every day before the first rain remains sunny, soThis geometric distribution has finite expected value. A first-step calculation gives $h_S=1+0.8h_S+0.2h_R$, with $h_R=0$, hence $h_S=5$ days. The same result follows from the geometric series ${\mathbb{E}}_S{T}_R={\sum }_{n\ge 0}0.8^n=5$. This is a mean waiting time, not a prediction that rain first occurs exactly five days later.

A probability row vector $\pi$ is a stationary distribution when $\pi P=\pi$. The Markov chain is reversible in that law when detailed balance holds: ${\pi }_i{P}_{ij}={\pi }_j{P}_{ji}$. Multiplying those equations along a finite path gives equal forward and reversed stationary path probabilities.

The Ehrenfest urn model moves one uniformly chosen ball between two boxes at each step. For $k\ge 1$ balls, the count in one box is a Markov chain on $\{0,\dots ,k\}$. Its stationary distribution is $2^{-k}\left(\genfrac{}{}{0ex}{}{k}{i}\right)$, and detailed balance makes it a reversible Markov chain. The chain has period two, but its mean recurrence time to the all-in-one-box state is still $2^k$.

If $r={\max }_{j\ge 2}|{\lambda }_j|$ for a finite irreducible Markov chain with at least two states, then its worst-case total variation distance from its stationary distribution satisfies $d(t)\ge r^t/2$. Choose a possibly complex eigenfunction $f$ for an eigenvalue of modulus $r$, normalize ${\max }_x|f(x)|=1$, and start where the maximum is attained. Since $\pi (f)=0$, $|P^{t}f(x)-\pi (f)|=r^t$; the defining sum for total variation distance bounds this difference by $2d(t)$. For $0<r<1$, this yields $t_{\mathrm{mix}}(\epsilon )\ge (r/(1-r))\log (1/(2\epsilon ))$ for $0<\epsilon <1/2$, because $-r\log r\le 1-r$.

For positive arrival rates and capacities with every route nonempty, maximizing $H_{\nu }(x)$ over $x\ge 0$, $Ax\le C$ has a unique positive optimizer $x^{\ast }$. The Karush-Kuhn-Tucker conditions give $x_r^{\ast }={\nu }_r{e}^{-(A^{T}z{)}_r}$ for nonnegative link prices with complementary slackness. Under proportional scaling by $N$, the Stirling formula gives a uniform logarithmic weight approximation $\log w_N(n)=N{H}_{\nu }(n/N)+O(\log N)$. Strict concavity and compactness give an objective gap away from $x^{\ast }$, while only polynomially many feasible states exist. Thus the stationary distribution has $n/N\to x^{\ast }$ in probability, with exponentially small probabilities outside any fixed neighborhood of $x^{\ast }$. In particular $\mathbb{E}[n]=N{x}^{\ast }+o(N)$.

For a fully observed Markov source, the Markov property makes the last state sufficient for the next symbol's conditional distribution. With stationary distribution $\pi$, the conditional entropy is thereforeusing $0{\log }_{2}0=0$. The chain rule for information entropy gives the exact finite-block identityDividing by $n$ proves the displayed entropy rate. The weighting by ${\pi }_i$ matters: rows with high uncertainty contribute in proportion to how often their states occur.

For an original asymmetric example, takeThe equality $\pi P=\pi$ verifies the stationary distribution. The single-symbol information entropy is $h_2(1/4)\simeq 0.81127812$, whereasThis formula describes observed states. If several states emit the same symbol, calculating the output entropy rate requires accounting for the hidden-state uncertainty; it is not generally the same row average.

For a finite irreducible Markov chain in continuous time, time averages converge almost surely to averages under its unique stationary distribution, from any initial state. Applying this to an activation indicator yields the corresponding throughput.

Consider the chosen finite minimization problemOnly adjacent integers are neighbors. At each interior state propose left or right with probability $1/2$ each; at an endpoint propose its sole neighbor with probability $1/2$ and itself with probability $1/2$. This proposal distribution is symmetric, including at the endpoints. The local minimum at $1$ is relative to these allowed moves; $4$ is the unique global minimum.

At $1$, improving-only local search rejects both neighbors because $E(0)=2$ and $E(2)=4$ exceed $E(1)=1$. For simulated annealing, choose $T_{\ast }=1/\ln 2$, using the natural logarithm. Conditional on proposing $2$ from $1$, the acceptance probability isThe one-step transition probability is instead $P_{T_{\ast }}(1,2)=(1/2)(1/8)=1/16$. Both subsequent rightward proposals are downhill, so the conditional probability of the precise next three states being $2,3,4$, given $X_0=1$ and fixed $T_{\ast }$, isThis is one possible escape route, not the probability of eventually reaching $4$. Rejections, excursions toward $0$, and repeated attempts give many other sample paths. The calculation establishes possibility, without claiming a general performance advantage from a selected sample path.

For comparison, halving $T_{\ast }$ makes the uphill acceptance probability $1/64$; doubling it makes that probability $1/(2\sqrt{2})$. Cooling makes it harder to leave this local minimum, even though the stationary distribution increasingly favors the global minimum. That tension is the central scheduling problem.

On a finite state space $F$ with finite objective function values, the fixed-$T$ target is a Boltzmann distribution:The partition function $Z_T$ normalizes the weights but cancels from the Metropolis–Hastings acceptance probability. A short detailed balance proof explains the target. With symmetric $q$ and $x\ne y$,Summing over $x$ gives ${\pi }_T{P}_T={\pi }_T$, so this is a stationary distribution. On a finite state space, a connected graph of positive symmetric proposals makes the accepted Markov chain an irreducible Markov chain. Convergence from any starting state additionally requires an aperiodic Markov chain; one positive self-transition suffices under irreducibility. Replacing $q$ by $(I+q)/2$ ensures this. For asymmetric $q$, require connectivity through bidirectional proposals: one-way proposals are rejected.^[ref]

In the five-state barrier example for simulated annealing, $T_{\ast }=1/\ln 2$ gives exact Boltzmann weights $2^{-E(x)}$. Multiplying by $16$ givesFor the uphill edge, detailed balance is visible numerically:Thus the global minimum has the largest individual stationary probability, $16/33$, but is far from certain at this positive $T_{\ast }$. At $T_{\ast }/2$ the exact stationary distribution is $(16,64,1,16,256)/353$.

For any finite $F$, let $M$ be the set of global minima and $E_{\min }$ the smallest objective function value. Cancelling $e^{-E_{\min }/T}$ from numerator and denominator shows that each nonminimal state's weight tends to zero as $T\downarrow 0$, while every state in $M$ has equal weight. Therefore ${\pi }_T$ tends to the uniform distribution on $M$. This is a statement about a family of equilibrium probability distributions; it does not prove that a cooling Markov chain follows that family.