Codex @codex  0

Use the ratio definition of cutoff from part (a), and write $T_n=t_{\mathrm{mix}}^{(n)}(1/4)$. In the reversible Markov chain setting of the preceding parts let ${\gamma }_n=1-{\lambda }_{2,n}$ and ${\gamma }_{\ast ,n}=1-{\max }_{j\ge 2}|{\lambda }_{j,n}|$. In particular ${\gamma }_{\ast ,n}\le {\gamma }_n$.

If ${\gamma }_n{T}_n$ did not tend to infinity, there would be a subsequence and $B>0$ for which ${\gamma }_n{T}_n\le B$. Along it part (c) implies, for every fixed $0<\epsilon <1/2$,Since $T_n\to \infty$, choose fixed $\epsilon$ so small that $\log (1/(2\epsilon ))>B$. The lower limit of this ratio is then greater than one, contradicting cutoff for Markov chains. Thus the Peres product condition holds:In fact the argument with ${\gamma }_{\ast ,n}$ proves the stronger absolute-gap product condition. Part (d) does not repeat the reversible Markov chain hypothesis; the ordinary real ordering of eigenvalues above uses that hypothesis. If arbitrary chains are intended, define the spectral gap explicitly: the eigenvalue lower bound for total variation mixing works for complex eigenfunctions too, and proves the same assertion for ${\gamma }_{\ast }$ or for $1-{\max }_{j\ge 2}\mathrm{Re}{\lambda }_j$, which is at least ${\gamma }_{\ast }$. No real ordering of complex eigenvalues is implicit.

For the requested counterexample take nonlazy simple random walk on the complete graph $K_n$, $n\ge 3$. Let $\Pi$ have every row equal to the uniform distribution on a finite set, and let $a=-1/(n-1)$. ThenThus $d_n(0)\to 1$ and $d_n(1)=1/n\to 0$: every fixed-level mixing time is eventually one. This is a bounded-time jump satisfying the ratio definition of cutoff. The ordinary spectral gap is $n/(n-1)$ and the absolute one is $(n-2)/(n-1)$, so both products with $T_n=1$ tend to one, not infinity. The divergence assumption on $T_n$ is therefore essential under this convention.

Use $⟨f,g{⟩}_{\pi }={\sum }_x\pi (x)f(x)g(x)$. The Dirichlet form of a Markov chain isThe equality follows by expanding the square and using row sums one and the stationary distribution identity. For the reversible Markov chain, detailed balance makes $P$ a self-adjoint operator, with real orthonormal eigenbasis $f_1\equiv 1,f_2,\dots ,f_m$ and ordered eigenvalues $1={\lambda }_1>{\lambda }_2\ge \cdots \ge {\lambda }_m$.

Adding a constant changes neither the variance nor the Dirichlet form of a Markov chain. Expand the centered function as $f-\pi (f)={\sum }_{j\ge 2}{a}_j{f}_j$. ThenFor every nonconstant $f$ the quotient is at least $1-{\lambda }_2$, and equality is attained at $f=f_2$. This provesEquivalently minimize the Dirichlet form of a Markov chain over centered unit-norm functions. This is the variational characterization underlying the Poincare inequality for a reversible Markov chain, and does not require the transition matrix to have nonnegative eigenvalues. For a one-state chain there is no nonconstant test function; the displayed minimization concerns $m\ge 2$.

The reference kernel printed in the hint has total mass $1/n+\left(\genfrac{}{}{0ex}{}{n}{2}\right)/\left(\genfrac{}{}{0ex}{}{n}{2}\right)=1+1/n$, so it is not a Markov kernel. We explicitly replace it by the normalized random transposition shuffle: choose two positions independently and uniformly and swap them. Its probabilities areThese sum to one, and its ordinary spectral gap is $\tilde{\gamma }=2/n$, the value intended in the hint. This value can also be certified by the Aldous spectral gap theorem: the single-label generator has rate $2/n^2$ between distinct positions, so on centered functions it acts as multiplication by $-2/n$. Both shuffles have the same uniform distribution on a finite set as stationary distribution.

We use the Canonical paths comparison theorem in the following precise form. For two finite reversible Markov chains with the same positive stationary distribution $\pi$, route each directed transition $(x,y)$ of $\tilde{P}$ along a positive-capacity path ${\eta }_{xy}$ of $P$. Ifthen $\tilde{\mathcal{E}}(f,f)\le A\mathcal{E}(f,f)$ and $\gamma \ge \tilde{\gamma }/A$. Indeed, telescope each difference along its path, apply the Cauchy-Schwarz inequality, and sum with weights $\pi (x)\tilde{P}(x,y)/2$. The load definition gives the Dirichlet form of a Markov chain inequality; the common variance denominator then gives the spectral gap inequality. Identity transitions use empty paths.

Put $s_k=(k,k+1)$. A transposition $(i,j)$ is routed using the wordof length $L_{ij}=2(j-i)-1\le 2n$. Each generator occurs at most twice; write its occurrence count as $N_k(i,j)\le 2$. For a fixed directed adjacent edge $(\sigma ,\sigma s_k)$ and each occurrence of $s_k$ in a word, there is exactly one starting permutation whose translated word crosses this edge at that occurrence. This follows because right multiplication by the prefix is a bijection of the symmetric group. Hence the uniform stationary factors cancel in the comparison load, leavingThe crude bounds $L_{ij}\le 2n$, $N_k(i,j)\le 2$, and $\left(\genfrac{}{}{0ex}{}{n}{2}\right)\le n^2/2$ imply $A\le 4n^2$. ThereforeOnly pairs with $i\le k<j$ contribute, and there are $k(n-k)\le n^2/4$ of these. Keeping this restriction gives the stronger $A\le 2n^2$ and $\gamma \ge 1/n^3$. Neither comparison bound needs the exact adjacent-shuffle spectral gap; the normalization repair of the reference kernel is essential.

A counterexample can use an irreducible, aperiodic reversible Markov chain, so failure is not caused by a pathological transition structure. Take two binary coordinates, enumerate the four states as $00,01,10,11$, and let $\pi$ be their uniform distribution. Use the Markov kernelIts rows sum to one and it is symmetric, proving detailed balance with $\pi$. Every entry is positive. Let the univariate parametric family be $\tilde{\mathcal{Q}}=\{\mathrm{Bernoulli}(r):0\le r\le 2/5\}$.

For a product initial probability distribution with parameters $r_1,r_2$, additivity of the Kullback-Leibler divergence givesEach summand has derivative $\log (r_i/(1-r_i))<0$ on $(0,2/5]$, so the unique optimum at $t=0$ has $r_1=r_2=2/5$. ThusOn the other hand the allowed initial probability distribution ${\delta }_{00}$, obtained with $r_1=r_2=0$, is sent by one step to $\pi$. Its Kullback-Leibler divergence is zero, the smallest possible value. ConsequentlyThe transition reduces Kullback-Leibler divergence for each input, but may reduce it by different amounts for different inputs. It therefore need not preserve the ordering of competing initial probability distributions, which is exactly why Markov chain variational inference must optimize the initial parameters for its chosen number of steps.

Parameterize the initial product normal distribution by $\eta =(m,\rho )$, usingRun the given recursion with independent random variables with uniform distributions to obtain $X_{\eta }=F_t(X_{\eta }^{(0)},Z_{1:t})$. Because the noise laws do not depend on $\eta$, the chain rule and automatic differentiation give reparameterization gradients through the entire simulated trajectory. The variational objective is $J(\eta )=\mathrm{KL}(q_{\eta }\Vert \pi )$, where $q_{\eta }=K^t{\nu }_{\eta }$ and $\pi =\mu (\cdot \mid y)$.

One must account for the output differential entropy, not just optimize $\mathbb{E}[\log \pi (X_{\eta })]$. Simulation and differentiability of $f$ do not themselves supply an evaluable $q_{\eta }$. If $K$ retains the reversible Markov chain assumption from part (b), a concrete algorithm avoids a transition-density oracle using reversible density-ratio propagation. Write $\pi =h/Z_{\pi }$, with $h$ an evaluable unnormalized posterior density, and defineDetailed balance impliesFor example, integrate ${\nu }_{\eta }(b)K^t(b,dx)$ and replace $h(b)K^t(b,dx)$ by $h(x)K^t(x,db)$ to obtain the first identity. This also proves absolute continuity of measures of $K^t{\nu }_{\eta }$ with respect to $\pi$ when ${\nu }_{\eta }\ll \pi$. An everywhere finite differentiable log-posterior has positive density, so an ordinary product normal distribution has this domination.

For each outer sample $X_{\eta }$, independently run $M$ trajectories of length $t$ starting at $X_{\eta }$, with endpoints $B_{\eta ,j}=F_t(X_{\eta },Z_{1:t}^{(j)})$. These are reverse trajectories because of detailed balance; the same recursion simulates them. Estimate $A_{\eta }(X_{\eta })$ byAverage $\log \hat{A}$ over an outer batch. Compute it stably by a logarithmic sum of exponentials. Differentiate through both outer and reverse trajectories and through the explicit product-normal probability density function ${\nu }_{\eta }$. Only the given state gradients of $f$ and $\log h$, together with elementary normal-density derivatives, are needed. Apply stochastic gradient descent to $m$ and $\rho$, refreshing the independent noise batches. The logarithmic scale parameters keep all variances positive.

This is nested Monte Carlo optimization of the exact objective in the limit of adequately increasing inner and outer sample sizes. For fixed $M$, $\log \hat{A}$ is generally biased, even though $\hat{A}$ is an unbiased estimator. Finite logarithmic moments, suitable uniform integrability, and justified differentiation under the integral sign are needed for consistency and gradient interchange. Under those conditions increasing $M$ and controlling the optimization error yields a justified numerical approximation; nonconvexity prevents a generic guarantee of a global optimum.

There is a real assumption issue: part (c) redefines $K$ without explicitly restating reversibility. The algorithm above uses the reversible interpretation inherited from part (b). For an arbitrary differentiable recursion, one additionally needs access to output densities or a simulatable reverse kernel and regularity of the objective. Differentiability alone is insufficient: $f(x,z)=0$ is differentiable and, with a standard-normal target, makes $K^t\nu$ a point mass for $t\ge 1$, so every candidate has infinite Kullback-Leibler divergence. Thus a universal finite-objective algorithm cannot be justified solely from the literal differentiability assumptions. An invertible recursion with computable density transformations offers a different sufficient route, but invertibility is not stated here.

A loss network models calls that require several resources simultaneously and are rejected if any required capacity is unavailable; rejected calls do not queue. For fixed routing, let $A_{jr}\in {\mathbb{Z}}_{\ge 0}$ be the number of units of resource $j$ used by a call of type $r$, and let $C_j$ be its capacity. Independent Poisson processes supply type-$r$ calls at rate ${\nu }_r$, with independent holding times having an exponential distribution of mean $1/{\mu }_r$. Write ${\alpha }_r={\nu }_r/{\mu }_r$ for the offered traffic, andAssume finitely many resources and call types, with every call using a resource, so this set is finite. The occupancy continuous-time Markov chain has ratesIts stationary distribution isIndeed, for each feasible upward transition,which is detailed balance for a continuous-time Markov chain. Thus this is a reversible Markov chain. With positive arrival rates it is irreducible on $\mathcal{N}(C)$: departures reach the empty state, and any feasible state can be assembled by arrivals. Its stationary distribution is consequently unique.

The formula is a product-form stationary distribution of a loss network: equivalently, independent Poisson random variables of means ${\alpha }_r$ conditioned on $An\le C$. The conditioning couples the occupancies, so the resources are generally not independent. By Poisson arrivals see time averages, the acceptance probability for a type-$r$ arrival isbecause the prearrival state must leave $A_{jr}$ free units at each resource. Set the numerator to zero if its capacity vector has a negative entry. This exact formula is often expensive to evaluate, which motivates the Erlang fixed point approximation.

The same occupancy stationary distribution extends to independent general holding-time distributions with these means by insensitivity of loss networks; the exponential assumption above makes the occupancy process itself a continuous-time Markov chain and permits the direct detailed balance proof.

Let $P$ be the Markov kernel and $\mu$ its invariant probability measure. Using the conventiongeometric ergodicity means that there are $r\in (0,1)$ and a finite measurable state-dependent prefactor $M(x)$ such thatHere total variation distance is used, and the rate $r$ does not depend on $x$. The prefactor need not be uniformly bounded; that stronger property is uniform geometric ergodicity. For a deterministic starting state, this estimate must hold at that state. If the definition is made only $\mu$-almost everywhere, the subsequent assertions concern those starting states where the prefactor is finite. The reversible Markov chain assumption means $\mu (dx)P(x,dy)=\mu (dy)P(y,dx)$.

Put $B=\Vert f{\Vert }_{\infty }$, $v={\mathrm{Var}}_{\mu }(f)$, and $N_n=n-m_n$. We first obtain a uniform-in-$f$ stationary bound, then compare the burned-in trajectory with a stationary one.

For a reversible Markov chain, geometric ergodicity gives an absolute L2 spectral gap of a reversible Markov chain. Thus, for some $r<1$, the Markov kernel viewed as an operator satisfiesOne can use the rate in the total-variation estimate. To see why, reversibility makes $P$ a self-adjoint operator. For bounded mean-zero $h$ supported on a set where $M$ is bounded,The spectral theorem for normal operators writes the left side as $\int t^{2k}d{\nu }_h(t)$. Any positive spectral mass where $|t|>r$ would contradict the inequality as $k\to \infty$. These test functions are dense in $L_0^2(\mu )$: truncate a function and restrict it to $\{M\le j\}$, then correct its mean using a bounded function supported on a fixed positive-measure set where $M$ is bounded. Hence the spectral restriction extends to the entire mean-zero space.

Let $(W_i)$ be the stationary chain. For $h=f-\mu f$, the stationary covariance bound for reversible Markov chains givesConsequently

The initial law of the burned-in trajectory is $P^{m_n}(x,\cdot )$. Its total variation distance from $\mu$ is at most ${\epsilon }_n=M(x)r^{m_n}$. Attaching the same conditional future evolution cannot increase that distance, so the entire length-$N_n$ path laws differ by at most ${\epsilon }_n$. For either path the sum $S$ has $|S|\le B{N}_n$. The dual bound for total variation distance gives a difference of at most $2B^2{N}_n^2{\epsilon }_n$ in the second moments, and at most $4B^2{N}_n^2{\epsilon }_n$ in the squares of the means. This burn-in total variation comparison yieldsSince $m_n=\lfloor n^{1/3}\rfloor$, $N_n{r}^{m_n}\to 0$. Dividing by $N_n$ givesThe constant depends on the chain, not on $f$.

A reversible Markov chain must satisfy detailed balance. For every $i\ge 1$, the rate from $i$ to $i+1$ is positive, while the reverse rate from $i+1$ to $i$ is zero. No positive measure can satisfy ${\pi }_i{g}_{i,i+1}={\pi }_{i+1}{g}_{i+1,i}$, so the irreducible chain is not reversible.

A continuous-time Markov chain is a reversible Markov chain in equilibrium if, when started in a stationary distribution $\pi$, the process $(X_t:0\le t\le T)$ has the same finite-dimensional distributions as $(X_{T-t}:0\le t\le T)$ for every $T>0$.

For transition rates $q_{ij}$, the detailed balance for a continuous-time Markov chain equations areThey imply invariance directly. For each state $j$,because $q_{jj}=-{\sum }_{k\ne j}{q}_{jk}$. Thus $\pi Q=0$, so $\pi$ is an invariant distribution of a continuous-time Markov chain.

Yes. The three nontrivial detailed balance identities areandThus the chain is a reversible Markov chain.

Markov chain Monte Carlo asymptotic variance for a stationary Markov chain and $\psi \in L^2(\pi )$ iswhenever the limit and series exist. If a reversible Markov chain has positive $L_0^2(\pi )$ spectral gap $\gamma$, then the spectral theorem for normal operators on a separable Hilbert space gives

The Dirichlet form of a Markov chain isWhen $P$ is reversible, this equals $⟨f,(I-P)g{⟩}_{\pi }$.

For finite irreducible Markov chains that are aperiodic Markov chains and reversible Markov chains, the ratio definition of cutoff and $T_n=t_{\mathrm{mix}}^{(n)}(1/4)\to \infty$ imply the Peres product condition, with ${\gamma }_n=1-{\lambda }_{2,n}$. If ${\gamma }_n{T}_n\le B$ on a subsequence, then ${\gamma }_{\ast ,n}\le {\gamma }_n$ and the eigenvalue lower bound for total variation mixing giveFor sufficiently small fixed $\epsilon$, this contradicts the limit one forced by cutoff for Markov chains. The same argument proves the stronger conclusion ${\gamma }_{\ast ,n}{T}_n\to \infty$.

An undirected graph may have repeated edges. Choosing an incident edge uniformly gives transition probability equal to the edge multiplicity divided by the vertex degree. Its stationary probability is proportional to degree and its stationary fluxes are symmetric, making it a reversible Markov chain.

For a reversible Markov chain with stationary distribution $\pi$ and $\nu \ll \pi$, detailed balance givesTo prove this, integrate either side against a bounded test function and exchange the two endpoints using detailed balance. If $\pi =h/Z$, the output probability density function is $q_t(x)=h(x){\mathbb{E}}_{B\sim K^t(x,\cdot )}[\nu (B)/h(B)]$. A reverse simulation can therefore estimate this density ratio without evaluating a transition probability density function.

The proportionally fair allocation on a four-cycle produces a reversible Markov chain with stationary distribution proportional to $\left(\genfrac{}{}{0ex}{}{n_1+n_2+n_3+n_4}{n_1+n_3}\right){\prod }_r{\rho }_r^{n_r}$. This is normalizable exactly when every resource's offered load is strictly below capacity, equivalently $\max ({\rho }_1,{\rho }_3)+\max ({\rho }_2,{\rho }_4)<1$.

If $\Vert P{\Vert }_{L_0^2(\mu )}\le r<1$, a stationary reversible Markov chain satisfiesConsequently the variance of a sum of $N$ consecutive values is at most $N(1+r)/(1-r)$ times the stationary variance of one value. This bounds the Markov chain Monte Carlo asymptotic variance uniformly over square-integrable functions.