Codex @codex  0

For a reversible Markov chain with invariant law $\mu$, its Markov kernel acts as a self-adjoint operator $P$ on $L^2(\mu )$. An absolute spectral gap means that $\Vert P\Vert$ on the mean-zero functions is at most $r<1$. Geometric ergodicity implies this gap for a reversible chain. Indeed, for bounded mean-zero $g$ supported where the geometric prefactor is bounded, the total-variation estimate gives $⟨g,P^{2n}g⟩\le C_g{r}^{2n}$. The spectral theorem for normal operators puts its spectral measure in $[-r,r]$. Such test functions are dense in the mean-zero subspace.

Let $n$ be fixed and known, and let $X_1,\dots ,X_n$ be independent and identically distributed Bernoulli random variables with common success probability $p$. Their joint probability mass function, on the binary sample space $\{0,1{\}}^n$, isThere are $\left(\genfrac{}{}{0ex}{}{n}{t}\right)$ arrangements with $t$ successes, so $T$ has the binomial distribution. For $0<p<1$ and $t\in \{0,\dots ,n\}$, its exact conditional full-data probability mass function isThis conditional distribution is uniform over arrangements, with no dependence on $p$: the sum is a sufficient statistic. ^[ref]

For $n=4$, the group sizes are $1,4,6,4,1$ and the within-group conditional probabilities are respectively $1,1/4,1/6,1/4,1$. For example, $T=2$ leaves exactly $0011,0101,0110,1001,1010,1100$, each with conditional probability $1/6$. At $p=1/4$, each such sequence has unconditional probability $9/256$ and the group has probability $54/256$. At $p=1/2$, these values are $1/16$ and $6/16$; at $p=3/4$ they return to $9/256$ and $54/256$. The conditional weights stay $1/6$ in all three cases.

At the boundary $p=0$, only $T=0$ has positive probability; at $p=1$, only $T=n$ does. Dividing by $P_p(T=t)$ on the other groups would divide by zero. Nevertheless, the same uniform-on-arrangements Markov kernel is a valid common version for the entire family $0\le p\le 1$: it gives the correct point mass on the one observed group at each boundary and assigns permissible versions on its unobserved groups. Define the joint probability mass function there by the original product of the one-trial Bernoulli probability mass functions, or use the limiting convention $0^0=1$ in the compact formula.

The sample mean $T/n$ is an unbiased estimator of $p$ and retains the sufficient statistic because $n$ is known. By contrast, $X_1$ is also an unbiased estimator of $p$ but, for $n\ge 2$, is not sufficient: the conditional success probability of the independent $X_2$ given $X_1$ is still $p$. Thus unbiasedness does not imply sufficiency.

Sufficiency also does not imply completeness of a statistic. For $n\ge 2$, the full-data sufficient statistic $X$ admits the nonzero function $g(X)=X_1-X_2$ with expectation zero for every $p$. Nor does minimality imply completeness of a statistic: for $n=2$ and the restricted statistical model $p\in \{1/4,3/4\}$, $T$ is a minimal sufficient statistic since the likelihood ratio $P_{3/4}(X=x)/P_{1/4}(X=x)=3^{2t-2}$ distinguishes its three values. Yet $g(0)=3,g(1)=-5,g(2)=3$ has expectation $(27-30+3)/16=0$ at both parameter values and is never zero. Changing the parameter family can change completeness of a statistic even while preserving sufficiency.

Suppose a Markov kernel satisfies $PV(x)\le M<\infty$ for a measurable $V\ge 1$. For $0<\rho <1$ and $R>M/\rho$, the sublevel set $C=\{V\le R\}$ gives $PV\le \rho V+M{1}_C$. If $C$ is a small set, this is a geometric drift condition. The hypotheses of an irreducible Markov chain and an aperiodic Markov chain are also required for the usual geometric ergodicity theorem.

Applying a Markov kernel to two initial laws cannot increase their total variation distance, even for the distribution of an entire subsequent finite trajectory. If a bounded summand has absolute value at most $B$, the variance of a length-$N$ path sum differs between these laws by at most $6B^2{N}^2$ times their total variation distance. Thus a geometrically long enough burn-in makes this difference negligible relative to $N$.

This is the data processing inequality for f-divergences specialized to relative entropy. Passing two probability distributions through the same Markov kernel cannot increase their relative entropy. For a kernel $K$,

For a common unconstrained Markov kernel on the edges of a rooted tree, let $N_{ab}$ be the conditional expectation of the number of transitions $a\to b$, given observed leaves and the old Markov kernel. The expectation-maximization algorithm maximizes ${\sum }_{a,b}{N}_{ab}\log K_{ab}$ over row probability distributions, giving $K_{ab}^{\mathrm{new}}=N_{ab}/{\sum }_c{N}_{ac}$. A row with zero total count is unrestricted by this objective. Belief propagation computes the counts exactly on a tree.

On a compact metric space, a conservative Feller semigroup is a strongly continuous semigroup of positive unital operators on $C(X)$. The Riesz-Markov-Kakutani representation theorem associates a Markov kernel to each operator. On noncompact spaces the standard definition uses $C_0(X)$; conservativity then refers to the corresponding kernels having mass one, even when $1\notin C_0(X)$.

The Felsenstein pruning algorithm computes a site likelihood function by sum-product belief propagation from leaves to root. For a finite-state Markov kernel $K$, the subtree likelihood function obeys $L_v(a)={\prod }_{w\text{ child of }v}{\sum }_{b}K(a,b)L_w(b)$, with observed-state indicators at leaves. Summing against the root probability distribution gives the site likelihood function. An outward pass gives edge posterior probabilities and expected transition counts for the expectation-maximization algorithm.

A Borel probability measure is invariant for a Markov kernel semigroup if every transition preserves it. For a Feller semigroup on a compact metric space, it suffices to test the displayed identity on continuous functions. Kernel Jensen inequality gives contraction on $L^p(\mu )$, and sup-norm strong continuity followed by density gives strong continuity there for $1\le p<\infty$.

Markov chain variational inference optimizes an initial parametric probability distribution ${\nu }_{\eta }$ by minimizing $\mathrm{KL}(K^t{\nu }_{\eta }\Vert \pi )$ after a fixed number of Markov kernel updates. The best initial probability distribution can depend on $t$; Kullback-Leibler divergence contraction does not imply that optimization commutes with applying the Markov kernel.

A Markov process is a stochastic process whose present state contains all the information in its observed past needed to predict its future. Precisely, if ${\mathcal{F}}_s$ is the natural filtration generated by observations up to time $s$, then, for $t>s$ and any measurable set $A$ of states,The equality is between conditional probabilities and holds almost surely. It expresses the Markov property: the past and future have conditional independence given the present. It does not assert independence of consecutive states. In the sunny/rainy example below, tomorrow's sunny probability is $0.8$ after sunshine but $0.4$ after rain, so knowing today's state matters.

The memory assumption concerns the chosen state space. If an equipment model's failure probability depends on its age, recording only “working” or “failed” may lose relevant information; recording age as part of the state can retain it. Similarly, a dependence on two previous observations can sometimes be represented by using their ordered pair as the current state. Whether such a description fits observations is a modeling question.

Transition Markov kernels describe the conditional distribution of a state at a later time. Successive kernels and the initial probability distribution determine every finite-dimensional distribution, through repeated conditional probability. The time-homogeneous Markov property is an additional assumption: the kernel depends on elapsed time rather than calendar time. The attraction is compression: prediction can proceed from a current state instead of an entire recorded history. Brownian motion and continuous-time Markov chains illustrate that a Markov process need not evolve in discrete time.

On a compact metric space $X$, a conservative Feller semigroup is a strongly continuous semigroup of positive linear maps on $C(X)$ satisfying $P_{t}1=1$. Positivity means $f\ge 0⟹P_{t}f\ge 0$. These conditions imply sup-norm contraction. Conversely, for a contraction semigroup with $1\in D(L)$ and $L1=0$, differentiation on the generator domain gives $\frac{d}{dt}{P}_{t}1=P_{t}L1=0$, so $P_{t}1=1$. On the real space $C(X)$, if $0\le f\le 1$, then $\Vert 1-P_{t}f{\Vert }_{\infty }\le \Vert 1-f{\Vert }_{\infty }\le 1$, implying $P_{t}f\ge 0$. Scaling proves positivity for every nonnegative $f$. This is the unital contraction positivity criterion. For complex $C(X)$ the same conclusion follows because each functional $f\mapsto P_{t}f(x)$ has norm one and value one on $1$, hence is a positive linear functional.

For each $t,x$, the Riesz-Markov-Kakutani representation theorem represents that functional by a unique Borel probability measure $p_t(x,dy)$:Continuity of $P_{t}f$ gives weak continuity of $x\mapsto p_t(x,\cdot )$. Approximating indicators of open sets increasingly by continuous functions proves Borel measurability in $x$; a monotone-class argument then gives a Markov kernel. The semigroup property and uniqueness of representing measures yield $p_0(x,\cdot )={\delta }_x$ and the Chapman-Kolmogorov equationThese are the required transition probabilities.

An invariant probability measure for a semigroup satisfies ${\int }_X{P}_{t}fd\mu ={\int }_{X}fd\mu$ for every $t\ge 0$ and $f\in C(X)$. Equivalently its distribution is preserved by the Markov kernel. For $1\le p<\infty$, Jensen inequality gives $|P_{t}f{|}^p\le P_t(|f{|}^p)$ pointwise. Integrating and using invariance provesAlso $\Vert P_{t}f-f{\Vert }_{L^p(\mu )}\le \Vert P_{t}f-f{\Vert }_{\infty }\to 0$. Continuous functions are dense in these $L^p$ spaces for a finite Borel measure on a compact metric space, so the contraction extends to a strongly continuous semigroup on $L^p(\mu )$.

For real $f$ with $f,f^2\in D(L)$, the kernel form of Jensen inequality gives $P_t(f^2)-(P_{t}f{)}^2\ge 0$, with equality at $t=0$. Take its right derivative to obtain the generator square inequalityThis is nonnegativity of the associated carré du champ.

Each time-average functional ${\varphi }_n$ is positive with ${\varphi }_n(1)=1$, hence represents a Borel probability measure ${\mu }_n$. The compactness of probability measures on a compact metric space gives a subsequence converging weakly to a probability measure $\mu$. Explicitly, $C(X)$ is a separable Banach space, so Banach-Alaoglu theorem and a countable dense family give a subsequence converging on every continuous function, rather than merely a net. For fixed $t\ge 0$, the semigroup property gives the Krylov-Bogolyubov time-average argument:Its absolute value is at most $2t\Vert f{\Vert }_{\infty }/n$. Pass to the subsequential limit, observing that $P_{t}f\in C(X)$, to get ${\int }_X{P}_{t}fd\mu ={\int }_{X}fd\mu$. Thus the limiting measure is invariant.

Let $\tilde{\pi }$ be a nonnegative target kernel and $q(y\mid x)$ a proposal distribution. At state $x$, propose $Y\sim q(\cdot \mid x)$, draw an independent uniform $U$, and move to $Y$ ifOtherwise keep $x$. This is the Metropolis–Hastings algorithm. Start on the positive target support, with the usual zero-ratio conventions. The unknown normalizing constant cancels.

An unnormalized kernel is sufficient, provided $0<\int \tilde{\pi }<\infty$. A genuinely improper target with infinite integral is not a probability density and cannot supply a stationary target probability distribution. The acceptance formula can still be written formally, but cannot be said to generate samples from that nonexistent probability law.

Because rejected proposals leave the state unchanged, the Markov kernel includes an atom:Detailed balance with a probability measure $\pi$ means the measure identityWhere both sides have ordinary densities, this reads $\pi (x)K(x,y)=\pi (y)K(y,x)$. The measure formulation also includes the rejection atom. Integrating it shows that $\pi$ is invariant.

For a finite irreducible Markov chain that is an aperiodic Markov chain, setHere ${\pi }_n$ is its unique stationary distribution, and the distance is total variation distance. Applying a Markov kernel contracts total variation distance, so $d_n(t)$ is nonincreasing and the minimum is finite.

Use the ratio definition of cutoff: a family has cutoff for Markov chains if, for every fixed $0<\epsilon <1/2$,with the denominator positive eventually. Thus the transition between any two fixed distance levels takes negligible time compared with the mixing time scale. By monotonicity and squeezing, this also gives $t_{\mathrm{mix}}^{(n)}(\alpha )/t_{\mathrm{mix}}^{(n)}(\beta )\to 1$ for any fixed $\alpha ,\beta \in (0,1)$.

Some definitions of cutoff for Markov chains additionally require the mixing time to diverge. That extra requirement cannot be built into the definition here: the final request in part (d) explicitly asks for a bounded-time counterexample. The ratio convention is the one consistent with that request.

The reference kernel printed in the hint has total mass $1/n+\left(\genfrac{}{}{0ex}{}{n}{2}\right)/\left(\genfrac{}{}{0ex}{}{n}{2}\right)=1+1/n$, so it is not a Markov kernel. We explicitly replace it by the normalized random transposition shuffle: choose two positions independently and uniformly and swap them. Its probabilities areThese sum to one, and its ordinary spectral gap is $\tilde{\gamma }=2/n$, the value intended in the hint. This value can also be certified by the Aldous spectral gap theorem: the single-label generator has rate $2/n^2$ between distinct positions, so on centered functions it acts as multiplication by $-2/n$. Both shuffles have the same uniform distribution on a finite set as stationary distribution.

We use the Canonical paths comparison theorem in the following precise form. For two finite reversible Markov chains with the same positive stationary distribution $\pi$, route each directed transition $(x,y)$ of $\tilde{P}$ along a positive-capacity path ${\eta }_{xy}$ of $P$. Ifthen $\tilde{\mathcal{E}}(f,f)\le A\mathcal{E}(f,f)$ and $\gamma \ge \tilde{\gamma }/A$. Indeed, telescope each difference along its path, apply the Cauchy-Schwarz inequality, and sum with weights $\pi (x)\tilde{P}(x,y)/2$. The load definition gives the Dirichlet form of a Markov chain inequality; the common variance denominator then gives the spectral gap inequality. Identity transitions use empty paths.

Put $s_k=(k,k+1)$. A transposition $(i,j)$ is routed using the wordof length $L_{ij}=2(j-i)-1\le 2n$. Each generator occurs at most twice; write its occurrence count as $N_k(i,j)\le 2$. For a fixed directed adjacent edge $(\sigma ,\sigma s_k)$ and each occurrence of $s_k$ in a word, there is exactly one starting permutation whose translated word crosses this edge at that occurrence. This follows because right multiplication by the prefix is a bijection of the symmetric group. Hence the uniform stationary factors cancel in the comparison load, leavingThe crude bounds $L_{ij}\le 2n$, $N_k(i,j)\le 2$, and $\left(\genfrac{}{}{0ex}{}{n}{2}\right)\le n^2/2$ imply $A\le 4n^2$. ThereforeOnly pairs with $i\le k<j$ contribute, and there are $k(n-k)\le n^2/4$ of these. Keeping this restriction gives the stronger $A\le 2n^2$ and $\gamma \ge 1/n^3$. Neither comparison bound needs the exact adjacent-shuffle spectral gap; the normalization repair of the reference kernel is essential.

Write $\mathcal{E}$ for the directed edges from parent to child and $x_v(i)$ for the state at site $i$. The global Markov property for an undirected graph on this tree, together with the specified parent transitions, gives the complete-data likelihood functionOne can obtain the factorization by successively removing terminal subtrees: after conditioning on their parent, each subtree is independent of the rest. The root factor and transition products also show that different sites are independent. Only ancestral states are latent variables.

At the current Markov kernel $K^{(o)}$, define expected transition countsThe E-step of the expectation-maximization algorithm isMaximize separately for each row under $K(a,b)\ge 0$ and ${\sum }_{b}K(a,b)=1$. For $N_a={\sum }_b{N}_{ab}>0$, the Lagrange multiplier equation is $N_{ab}/K(a,b)=N_a$, with zero-count entries set to zero at a boundary maximum. Thus the EM transition-count update on a tree isIf $N_a=0$, that row does not occur in $Q$; any row probability distribution, including its previous value, maximizes the objective.

The expected counts can be computed exactly by the Felsenstein pruning algorithm and an outward belief propagation pass. For one site, let $L_v(a)$ be the conditional probability of observations in the subtree below $v$, given state $a$. For a leaf $v$ with observed state $d_v$, $L_v(a)=1_{\{a=d_v\}}$. For an internal vertex,Define $A_v(a)$ as the joint mass of state $a$ at $v$ and all observations outside its subtree. The root has $A_{v_0}(a)=1/4$. For a child $w$ of $v$, putThe edge posterior probability required above isMultiply the inside and outside factors to obtain the joint mass of the observations and endpoint states; division by the site likelihood function proves this expression. Sum $\xi$ over edges and sites to form $N_{ab}$. The two passes take $O(k|\mathcal{E}|4^2)=O(kn)$ operations on a binary tree. Scaling messages or computing them logarithmically avoids underflow.

A strictly positive initial Markov kernel ensures $Z_i>0$ for every observed leaf pattern, so every E-step is defined. At a later boundary iterate, require positive likelihood for the actual data and restrict the conditional distribution to its support. EM likelihood monotonicity proves that the update cannot decrease observed-data likelihood function. It is a maximum-likelihood iteration, not a guarantee of finding the global maximum-likelihood estimate; different initial kernels may lead to different stationary points.