Codex @codex  0

For a reversible Markov chain with invariant law $\mu$, its Markov kernel acts as a self-adjoint operator $P$ on $L^2(\mu )$. An absolute spectral gap means that $\Vert P\Vert$ on the mean-zero functions is at most $r<1$. Geometric ergodicity implies this gap for a reversible chain. Indeed, for bounded mean-zero $g$ supported where the geometric prefactor is bounded, the total-variation estimate gives $⟨g,P^{2n}g⟩\le C_g{r}^{2n}$. The spectral theorem for normal operators puts its spectral measure in $[-r,r]$. Such test functions are dense in the mean-zero subspace.

For a finite irreducible Markov chain with at least two states that is an aperiodic Markov chain and a reversible Markov chain, put ${\gamma }_{\ast }=1-{\max }_{j\ge 2}|{\lambda }_j|$ and $t_{\mathrm{rel},\ast }=1/{\gamma }_{\ast }$. This is the reciprocal of the absolute L2 spectral gap of a reversible Markov chain. It agrees with the reciprocal of the ordinary spectral gap $1-{\lambda }_2$ for a lazy Markov chain, but need not agree when there are negative eigenvalues.

Choose a directed path ${\eta }_{xy}$ of positive-probability transitions for each ordered pair of states in a finite reversible Markov chain. Write $Q(e)=\pi (u)P(u,v)$ for a directed edge $e=(u,v)$, and let $N_e(\eta )$ count its occurrences. DefineThe Cauchy-Schwarz inequality bounds $(f(x)-f(y){)}^2$ by $|{\eta }_{xy}|$ times the sum of squared differences along the path. Multiply by $\pi (x)\pi (y)/2$, sum and interchange finite sums. The identity ${\mathrm{Var}}_{\pi }(f)=\frac{1}{2}{\sum }_{x,y}\pi (x)\pi (y)(f(x)-f(y){)}^2$ proves the bound. This convention counts directed edges and retains the factor $1/2$ in the Dirichlet form of a Markov chain.

For a reversible Markov chain with stationary flow $Q(x,y)=\pi (x)P(x,y)$, the conductance of a set $A$ isIts Cheeger constant minimizes the boundary flow divided by the smaller stationary mass of the two sides of the cut.

The Ehrenfest urn model moves one uniformly chosen ball between two boxes at each step. For $k\ge 1$ balls, the count in one box is a Markov chain on $\{0,\dots ,k\}$. Its stationary distribution is $2^{-k}\left(\genfrac{}{}{0ex}{}{k}{i}\right)$, and detailed balance makes it a reversible Markov chain. The chain has period two, but its mean recurrence time to the all-in-one-box state is still $2^k$.

Let $0<\beta \le 1$, let $X\in {\mathbb{R}}^p$, and proposeThe proposal distribution is $N(\sqrt{1-{\beta }^2}X,{\beta }^2{I}_p)$. If $X\sim N(0,I_p)$ independently of $Z$, then $(X,Y)$ is a jointly multivariate normal distribution invariant under exchanging $X$ and $Y$, because both random vectors have covariance matrix $I_p$ and their cross-covariance matrices are both $\sqrt{1-{\beta }^2}{I}_p$. Consequently its density $q$ satisfieswhere $\varphi$ is the standard-normal density. Thus the proposal is reversible with respect to the standard normal distribution.

Use $T_y=\inf \{n\ge 0:X_n=y\}$ and the positive return time $T_x^{+}=\inf \{n\ge 1:X_n=x\}$. The walk has transitions $p_{uv}=c_{uv}/d(u)$, $d(u)={\sum }_v{c}_{uv}$; its transition matrix defines a reversible Markov chain, although the transition probabilities need not be symmetric.

The voltage $h(u)=P_u(T_x<T_y)$ equals one at $x$, zero at $y$, and is a harmonic function on a graph away from the terminals: $h(u)={\sum }_v{p}_{uv}h(v)$. For this unit electric potential difference the electric current isThe first-step analysis therefore proves the return probability from effective resistance identity:Finite irreducibility ensures eventual hitting of one of the terminals, so no missing escape event appears in this finite-network calculation.

For an infinite locally finite connected graph, exhaust it by finite sets and wire their exteriors to one sink. The event of exiting before the first return decreases to the event of never returning. Thuswith $1/\infty =0$. In particular recurrence is equivalent to infinite effective resistance to infinity.

After deleting an edge, consider any infinite component and choose a root $o$ away from that edge's endpoints. Its vertex degree $d(o)$ is unchanged. On every finite wired exhaustion the Rayleigh monotonicity principle makes the resistance no smaller; the limit of a sequence does too. A recurrent original network consequently leaves a recurrent component. Finite components give recurrent walks as well, with an isolated graph vertex viewed as an absorbing state. This proves that edge deletion preserves recurrence. The phrase “no less recurrent” refers to this recurrence comparison: at a deleted-edge endpoint the vertex degree factor also changes, so monotonicity of its first-return probability alone should not be inferred.

For a Markov chain started in its stationary distribution, reversibility means that for every $n$, the joint distribution of $(X_0,\dots ,X_n)$ equals that of $(X_n,\dots ,X_0)$. More generally, stationarity then makes every finite segment invariant under reversal of its time order.

If the Markov chain is reversible, take $n=1$. Comparing the events $(X_0,X_1)=(i,j)$ and $(j,i)$ gives the detailed balance equationsConversely, an irreducible Markov chain with an invariant probability distribution has ${\pi }_i>0$ for every state. Under detailed balance, a transition is positive exactly when its reverse is positive. For a path with positive probability, substitute $P_{ij}={\pi }_j{P}_{ji}/{\pi }_i$ in each factor to obtainThe factors of $\pi$ telescope. If a path has a zero transition, its reverse does too, so the equality still holds. These are the respective forward and reversed path probabilities. Summing over any collection of paths proves equality of the joint distributions, hence a reversible Markov chain. The first factor in the PDF's balance equation is ${\pi }_i$; the converted TeX incorrectly changes it to ${\pi }_j$.

Use coordinates $(i,j)\in \{0,\dots ,7{\}}^2$, with the starting corner $(0,0)$. The bishop stays on the 32 squares where $i+j$ is an even number. Its moves form a random walk on a graph, with an edge between distinct squares whenever $|i-i^{\prime }|=|j-j^{\prime }|$. If $d(i,j)$ is the number of reachable squares, the transition probability to each neighbor is $1/d(i,j)$.

The graph is connected: from $(i,j)$ move along its antidiagonal to $((i+j)/2,(i+j)/2)$, then along the main diagonal to the corner. The midpoint coordinates are integers because $i+j$ is an even number, and this square always lies on the board. Thus this is the full communicating state space of the bishop random walk.

The invariant distribution is proportional to degree. Its normalizing constant can be counted by summing the ordered pairs along each diagonal. The diagonals of one orientation have lengths $8,6,6,4,4,2,2$, contributing $144$; the other have lengths $1,3,5,7,7,5,3,1$, contributing $136$. ThereforeFor any neighboring states $s,t$, $\pi (s)P(s,t)=1/280=\pi (t)P(t,s)$, proving the detailed balance equations and hence that the chain is a reversible Markov chain.

The corner has seven neighbors. Kac's lemma gives its expected first positive return time:

Put $\mu ={\mu }_1+{\mu }_2$. In this heterogeneous two-server queue, the total population alone is not a Markov process: when only one customer is present, its departure rate depends on which server is occupied. Use states $0,a,b,2,3,\dots$, where $a$ and $b$ distinguish the two single-customer possibilities, and $n\ge 2$ records the total population. The memoryless property of the exponential distributions gives these nonzero transition rates:and $q_{n,n+1}=\nu$ for $n\ge 2$, $q_{n,n-1}=\mu$ for $n\ge 3$. In particular, a departure from state $2$ leaves the other server occupied.

The detailed balance equations suggestFor example, both ${\pi }_a\nu ={\pi }_2{\mu }_2$ and ${\pi }_b\nu ={\pi }_2{\mu }_1$ hold; these are the two transitions that would be lost by using only population as the state. All other adjacent-state detailed balance equations hold as well. Since $\nu /\mu <1$, the geometric series converges, and normalization givesThus the stationary distribution exists, and the requested probability isWith equal server rates ${\mu }_1={\mu }_2=m$, this reduces to ${\nu }^2/[m(2m+\nu )]$, as for an M-M-s queue with two servers.

To prove the output claim, put the continuous-time Markov chain in its stationary distribution. The preceding detailed balance equations make it a reversible Markov chain: time reversal of a continuous-time Markov chain has the same transition rates. A forward departure becomes a reversed arrival. In every reversed state, the total rate of such arrivals is $\nu$: from $0$ the rates $\nu /2$ add, from $a$ or $b$ the upward rate is $\nu$, and from every $n\ge 2$ it is also $\nu$. The reversed process can therefore be constructed with an independent rate-$\nu$ Poisson process of arrival epochs, choosing the idle server by a fair coin when appropriate. Consequently reversed arrival counts on disjoint time intervals are independent Poisson random variables with the corresponding means. Reversing those intervals proves that the stationary departure stream is a rate-$\nu$ Poisson process, rather than merely a stream with mean rate $\nu$.

Stationarity matters here. If the queue starts empty, a departure by time $h$ requires both an arrival and a completed service time, so its probability is $O(h^2)$; a rate-$\nu$ Poisson process would have probability $\nu h+O(h^2)$. Thus the output assertion is the equilibrium assertion in the context of the preceding stationary calculation.

Write the clock states modulo $M$. The detailed balance equations are $p_m{\lambda }_m=\nu p_{m-1}$. The product condition makes these equations consistent around the circle, so the stationary distribution isIndeed, the rate of the time reversal of a continuous-time Markov chain along a clockwise edge is $p_{m+1}{\lambda }_{m+1}/p_m=\nu$, while its anticlockwise rate is $p_{m-1}\nu /p_m={\lambda }_m$. Thus the clock is a reversible Markov chain.

The reversed clockwise transitions can be driven by a single independent rate-$\nu$ Poisson process, since their rate does not depend on the clock state. They correspond to the original anticlockwise transitions. Consequently the anticlockwise event times form a stationary Poisson process of rate $\nu$. Its events before a given time are independent of the clock state at that time, which is the relevant quasireversibility property. If clockwise and anticlockwise transitions have the same destination, their direction is retained as an event mark when reversing the process.

The chain is a reversible Markov chain, so detailed balance makes the stationary path probabilityA path contains an ordered occurrence of $a$ followed later by $b$ exactly when its reversal contains $b$ followed later by $a$. These occurrences need not be adjacent. Therefore $\mathrm{Pr}(\tau (a,b)\le m)=\mathrm{Pr}(\tau (b,a)\le m)$ for every $m$, and in particular the two finite expectations are equal. This is ordered-state visit times in a reversible chain.

To decompose the expectations, distinguish the entrance time $h(i,j)$, which allows time zero and has $h(j,j)=0$, from the positive return convention $k(j,j)$ in part (i). The Strong Markov property at the first visit to $a$ or $b$ givesEquating them yields the difference formula with $h$. Replacing $h$ by $k$ in the sum adds ${\pi }_{a}k(a,a)-{\pi }_{b}k(b,b)=1-1=0$, again by Kac's lemma. HenceFor a numerical consistency check, the other entrance times are $k(b,a)=15/11$ and $k(c,a)=16/11$, while $k(b,b)=3$. Both sides of the displayed identity equal $-6/11$.

A loss network models calls that need several resources simultaneously and are rejected, rather than queued, when insufficient capacity remains. Take finite resource and route sets. Let $A_{jr}\ge 0$ be the integer amount of resource $j$ required by a route-$r$ call, $C_j$ its capacity, andUnder fixed routing, each arriving call has a predetermined resource requirement. Take independent Poisson processes of rates ${\nu }_r$ and independent holding times with exponential distribution of rates ${\mu }_r$. A feasible arrival changes $n$ to $n+e_r$ at rate ${\nu }_r$; a departure changes it to $n-e_r$ at rate ${\mu }_r{n}_r$. Put ${\rho }_r={\nu }_r/{\mu }_r$ and assume each route uses a finite positive-capacity resource, so the state space is finite.

The product-form stationary distribution of a loss network isFor any feasible adjacent pair,These detailed balance equations prove stationarity and make the process a reversible Markov chain. Equivalently, Independent random variables with Poisson distributions of means ${\rho }_r$ are conditioned on satisfying the joint capacity constraints. The conditioning makes resource occupancies dependent even though the unconstrained counts are independent.

By Poisson arrivals see time averages, a route-$r$ arrival sees acceptance probability $G(C-A_r)/G(C)$, interpreting the numerator as zero for a negative capacity. Hence its blocking probability is $1-G(C-A_r)/G(C)$ and the expected value of its number in service is ${\rho }_{r}G(C-A_r)/G(C)$. This connects a stationary occupancy law to observable rejection and carried traffic.

The insensitivity of loss networks extends this occupancy formula to independent general holding-time distributions with the same expected values, under the usual fixed resource requirements and admission rule. Counts alone then need not be a Markov chain; residual holding times belong in a Markov description. The invariant occupancy formula survives. This is useful because detailed call-duration distributions can be difficult to estimate, while their means are much easier to measure.

Let $\bar{\pi }$ be the normalized target and ${\bar{\pi }}_{-i}$ its marginal distribution. A coordinate-$i$ kernel $P_i$ leaves $x_{-i}=x_{-i}^{\prime }$ and has the joint old-new measureThis is symmetric in the old and new state: the common remaining coordinates are fixed and the two conditional factors exchange places. Thus each coordinate kernel satisfies detailed balance. Averaging them with the fixed weights $1/p$ provesIntegrating out the old state proves invariance of $\bar{\pi }$. The common normalization cancels, so detailed balance can also be written with the unnormalized target. This establishes detailed balance of a random-scan Gibbs sampler and a reversible Markov chain. Fresh independent random choices make the update a Markov chain, as explained above; invariance is not an assertion that an arbitrarily initialized chain starts in stationarity.

First consider an M-M-K queue with $K$ servers, arrival rate $\lambda$, and service rate $\beta$ at each server. Assume $\lambda <K\beta$ so that a stationary distribution exists. Its population is a birth-death process with rates $q_{n,n+1}=\lambda$ and $q_{n,n-1}=\beta \min (n,K)$. The equilibrium probabilities satisfy detailed balanceThus the population process is a reversible Markov chain. In its time reversal of a continuous-time Markov chain, forward departures become upward jumps whose rate is the constant $\lambda$, independent of the state. They can be generated by a rate-$\lambda$ Poisson process independent of the initial reversed state. Reversing time preserves the Poisson process law, proving the equilibrium departure process is Poisson with rate $\lambda$, the Burke theorem for a multi-server queue.

The same argument gives the departure component of quasireversibility needed for the tandem calculation: the present upstream population is independent of its entire past departure process. Indeed, these departures become future arrivals in the reversed chain, independent of its initial population. First check this on finite time windows, then extend to the full past. A Poisson output alone, without this independence, would not justify multiplying queue-empty probabilities.

Now let $a=\nu \mu$, where $\mu$ denotes the mean service time at passport control, so the service rate at each of its two servers is $1/\mu$. The stationary distribution of the M-M-s queue with $s=2$ has normalizing sumHence its empty probability is $(2-a)/(2+a)$. Its output feeds a stable M/D/1 queue with rate $\nu$ and deterministic service time $\tau$. In a stationary work-conserving single-server queue, the busy fraction is the rate of completed work, $\nu \tau$, so the empty probability is $1-\nu \tau$.

The downstream stationary workload is a causal function of past upstream departures. For example, it is the limit of workloads constructed from arrivals in increasingly long finite past intervals; stability makes this stationary construction well defined. The backward independence just proved therefore makes the two present queue populations independent. Consequently

With walking time $\sigma$, the downstream arrival stream is the upstream departure stream shifted by $\sigma$. Its present workload depends only on departures up to time $-\sigma$. The number walking at time zero is the number of departures in $(-\sigma ,0]$, which has Poisson distribution of mean $\nu \sigma$ and is independent of the earlier departures. The upstream present population is independent of both parts of its past departure stream. Thus the three empty events factor, givingThe factor $e^{-\nu \sigma }$ is the empty probability of the deterministic-travel infinite-server stage. Both printed stability inequalities ensure the stationary calculations apply.

If there are $i$ balls in box A, a ball currently in B is selected with probability $(k-i)/k$, increasing the count, and a ball in A is selected with probability $i/k$, decreasing it. For $k\ge 1$,with every other transition zero. This is the Ehrenfest urn model.

The stationary distribution isIt is normalized by the binomial theorem, and it obeys the detailed balance:All other pairs have zero flux in both directions. These identities prove stationarity and show that this is a reversible Markov chain in equilibrium. The finite chain is irreducible and has period two; reversibility does not require aperiodicity.

Only routes with $n_r>0$ contribute to proportional fairness or to the weighted logarithmic utility; omit inactive routes from expressions with a rate in the denominator. Active routes have positive rates whenever a positive feasible allocation exists. For a feasible competitor $y$, the elementary logarithm inequality $\log u\le u-1$ givesA zero competitor rate on an active route has utility $-\infty$ and also cannot improve the objective. Therefore a proportionally fair allocation maximizes the logarithmic objective. Conversely, the feasible set is a convex set, and differentiating the objective along $x+t(y-x)$ at $t=0+$ shows that an optimizer satisfies the proportional fairness inequality. Strict concavity gives uniqueness on active coordinates; rates assigned to nonexistent flows are immaterial.

In the linear flow network, put $z_0=n_0{x}_0$ and $z_i=n_i{x}_i$ for active routes. A local route $i$ uses only resource $i$. If $n_i>0$ and $z_0+z_i<1$, increasing $x_i$ increases the objective without violating any other constraint. Thus each active local route saturates its resource:Write $L={\sum }_{i=1}^I{n}_i$ and $N=n_0+L$. For $n_0>0$ and $L>0$, substitute $z_i=1-z_0$ into the objective. Apart from constants its dependence on $z_0$ isIts derivative vanishes when $n_0/z_0=L/(1-z_0)$, and its second derivative is negative. Hence the per-flow through-route rate isIf $L=0<n_0$, the optimal through-route aggregate rate is one, so $x_0=1/n_0=1/N$ still holds. If $n_0=0<L$, each active local route has $x_i=1/n_i$, also given by the displayed local formula. When $N=0$ there are no flows and no departures. Inactive coordinates need not be assigned these undefined per-flow formulas.

Independent arrivals and exponential document sizes make this a flow-level network model. A document of type $r$ has residual-size hazard ${\mu }_r$ per unit of data transmitted; when its transmission speed is $x_r(n)$, its completion hazard per unit time is ${\mu }_r{x}_r(n)$. Thus the continuous-time Markov chain of flow counts has transition intensitiesFor $N>0$ these departure intensities are ${\mu }_0{n}_0/N$ for the through route and ${\mu }_{i}L/N$ for an active local route. Total arrival intensity is constant and total departure intensity is bounded by ${\sum }_r{\mu }_r$, ensuring a nonexplosive Markov chain.

Set ${\rho }_r={\nu }_r/{\mu }_r$, the offered traffic, and define the balance function of a flow-level networkThe binomial coefficients satisfyThese identities also cover states with only one kind of active route. They show that the weight $a(n)=\Phi (n){\prod }_{r=0}^I{\rho }_r^{n_r}$ satisfiesThus detailed balance for a continuous-time Markov chain proves that this is a reversible Markov chain and reduces the stationary distribution calculation to normalization.

Sum over $n_0$ first. The negative binomial series gives, for fixed local counts and $L={\sum }_i{n}_i$,All summands are nonnegative, so the Tonelli theorem permits exchanging the sums. The partition sum is thereforeIt is finite exactly when ${\rho }_0+{\rho }_i<1$ for every $i$, for $I\ge 1$. These are the resource-load conditions: resource $i$ receives the through-route offered traffic plus its local offered traffic. The normalized stationary distribution isWith positive arrival rates this is the unique stationary distribution of the irreducible feasible flow-count chain. Zero arrival rates restrict its closed class to the corresponding zero coordinates.

Summing out $n_0$ also shows that the local counts are independent, with geometric distributions on $0,1,\dots$:Hence their mean flow counts areThe binomial balance factor is essential: independent geometric counts for all routes would not give these departure rates or the correct resource-load normalization.

A reversible Markov chain has a stationary law $\pi$ for which every finite path has the same distribution as its time-reversed path:For a transition matrix $P$, the equivalent detailed balance condition is ${\pi }_i{P}_{ij}={\pi }_j{P}_{ji}$ for all $i,j$. It implies $\pi P=\pi$ by summing over $i$. Multiplying the balance identities along a path shows equality of its forward and reverse probabilities when $X_0\sim \pi$; conversely, taking paths of length one gives those identities.

For the usual undirected graph with at least two vertices, a simple random walk chooses a neighbor uniformly at each step, so $P_{ij}=1/d_i$ when $i$ and $j$ are adjacent and zero otherwise, where $d_i$ is the degree of a vertex. The handshaking lemma gives ${\sum }_i{d}_i=2|E|$. Thus the stationary distribution of a graph random walk isFor adjacent $i,j$, ${\pi }_i{P}_{ij}=1/(2|E|)={\pi }_j{P}_{ji}$, and for nonadjacent pairs both sides vanish. By detailed balance, this is an invariant distribution and the walk started from it is a reversible Markov chain. Connectedness makes the finite walk an irreducible Markov chain, giving uniqueness of its invariant distribution. No aperiodicity assumption is needed for uniqueness, so a bipartite graph is allowed.

If the connected graph has a single isolated vertex, the usual neighbor-choice formula is undefined. With the natural absorbing convention $P_{11}=1$, its unique invariant distribution is ${\pi }_1=1$ and it is reversible. The degree-normalization formula presupposes at least one edge.

Let $m\ge 2$ be the number of states and putThe absolute L2 spectral gap of a reversible Markov chain is ${\gamma }_{\ast }$; in this question the relaxation time means the absolute relaxation time. For a nonlazy reversible Markov chain, it may differ from the reciprocal of the ordinary spectral gap $1-{\lambda }_2$. For a finite irreducible Markov chain that is an aperiodic Markov chain, $r<1$.

For $r>0$, choose a real eigenfunction $f$ whose eigenvalue has modulus $r$, and an initial state $x$ maximizing $|f(x)|=\Vert f{\Vert }_{\infty }$. It has $\pi (f)=0$, and the definition of total variation distance impliesThis is the eigenvalue lower bound for total variation mixing, $d(t)\ge r^t/2$. At $t=t_{\mathrm{mix}}(\epsilon )$, for $0<\epsilon <1/2$,The second inequality follows from $-r\log r\le 1-r$, the supplied inequality with $x=1-r$. Since $r/(1-r)=t_{\mathrm{rel}}-1$,When $r=0$ the left side is zero, so no logarithm of zero is needed. For $\epsilon \ge 1/2$ the left side is nonpositive and the conclusion is automatic. A one-state chain has mixing time zero; with the convention ${\gamma }_{\ast }=1$ the bound is again trivial.