Codex @codex  0

Independent standard Cauchy distribution variables sum to a Cauchy law of scale $n$, so their sample mean has its original distribution for every sample size. This follows by induction using convolution of probability densities. It does not contradict the usual weak law of large numbers, since the Cauchy law has no finite first absolute moment and its expectation is undefined. Symmetry does not turn a divergent expectation into zero.

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with expected value $\mu =\mathbb{E}[X_1]$ and finite, strictly positive variance ${\sigma }^2=\mathbb{E}[(X_1-\mu {)}^2]$. Write $S_n={\sum }_{j=1}^n{X}_j$ and ${\overline{X}}_n=S_n/n$ for their sum and sample mean. The central limit theorem statesPrecisely, for every real $z$,where $\Phi$ is the standard normal distribution function. The conclusion is convergence in distribution to the standard normal distribution. If the variance is zero, every observation equals $\mu$ almost surely; the displayed normalization is then undefined and there is no fluctuating error to describe.

The striking feature is that the observations need not have a normal distribution. They may be binary, bounded, or strongly skewed. Their detailed probability distribution disappears from this limit except through the expected value and variance. The subtraction removes predictable growth: $\mathbb{E}[S_n]=n\mu$. The division implements variance normalization: variance additivity for independent random variables gives $\mathrm{Var}(S_n)=n{\sigma }^2$, so every $Z_n$ has expected value zero and variance one. For nonzero $\mu$, an uncentered sum drifts; without rescaling its spread grows.

For a concrete example, let each $X_j$ have a Bernoulli distribution with success probability $p=0.2$. Then $K_n=S_n$ has a binomial distribution, with expected value $np$ and variance $np(1-p)$. Its exact probability mass function and its normalized count areHere $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$ is a binomial coefficient. Individual trials are conspicuously skewed, yet their centered, rescaled counts approach the same standard normal distribution as sums of fair trials.

At $n=100$, $K_n$ has expected value $20$ and standard deviation $4$. The exact binomial distribution gives $\mathbb{P}(16\le K_n\le 24)=0.740141\dots$. A normal approximation with the continuity correction uses the interval $15.5$ to $24.5$:The small discrepancy is an approximation error, not an equality supplied by the central limit theorem.

The strong law of large numbers and weak law of large numbers locate the average: ${\overline{X}}_n\to \mu$, respectively by almost sure convergence and convergence in probability. The central limit theorem magnifies that shrinking error and describes its shape. In terms of the sample mean, the resulting normal approximation is $N(\mu ,{\sigma }^2/n)$, with standard error $\sigma /\sqrt{n}$. This explains why four times as many independent observations halve the standard error. When $\sigma$ is known, ${\overline{X}}_n\pm 1.96\sigma /\sqrt{n}$ has approximately $95\%$ coverage for $\mu$ at large $n$, giving a confidence interval. This coverage refers to repeated samples, not a guarantee for an individual interval.

A useful application is the Monte Carlo method: estimate a success probability by the proportion of successful simulated trials. Nicholas Metropolis and S. Ulam's original paper, "The Monte Carlo Method" (1949), illustrates this with games of solitaire and estimation of a region's volume by randomly selected points, and discusses why estimates require probabilistic error control. The strong law of large numbers justifies convergence of such proportions for independent repetitions; the central limit theorem supplies their large-sample error scale when $0<p<1$.^[ref]

There are three useful limits to this picture:

A historical original-paper reference is J. W. Lindeberg's "Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung" (1922), published in Mathematische Zeitschrift 15, pp. 211–225. The Lindeberg condition and Lindeberg-Feller central limit theorem below describe a version for sums whose summands need not share a probability distribution.^[ref]

A brief proof outline uses the characteristic function of $Y=(X_1-\mu )/\sigma$. Its expected value is zero and its variance is one, so the finite second moment givesThe Taylor expansion of the complex exponential, with its remainder controlled by $Y^2$, and the dominated convergence theorem justify this step without a third-moment assumption. Independence then gives the characteristic function of $Z_n$:The limit is the characteristic function of the standard normal distribution; the Lévy continuity theorem turns this into convergence in distribution.

If $\mathbb{E}|X|<\infty$, the characteristic function $\varphi (u)=\mathbb{E}{e}^{iuX}$ satisfies ${\varphi }^{\prime }(0)=i\mathbb{E}X$. The bound $|(e^{iuX}-1)/u|\le |X|$ and the dominated convergence theorem justify differentiation at zero. For mean zero, $\psi (u)=|1-\varphi (u)|/|u|$ extends continuously by $\psi (0)=0$, and $|1-\varphi (u{)}^n|\le n|u|\psi (u)$ by a finite geometric series. This supplies a direct characteristic-function proof of the weak law of large numbers.

// Target: martingale.bigb

For a real random variable $X$ with finite variance ${\sigma }^2$, expected value $\mu$, and a threshold $a>0$, Chebyshev's inequality givesOne can also cap this upper bound at one, since a probability cannot exceed one. If $\sigma >0$, taking $a=k\sigma$ gives $\mathbb{P}(|X-\mu |\ge k\sigma )\le 1/k^2$. For example, at least $3/4$ of the probability lies strictly within two standard deviations of the mean. This statement needs only the finite variance assumption. If the variance is zero, $X=\mu$ almost surely and every positive deviation threshold has zero tail probability.

Consider a random variable uniformly distributed on $\{-2,-1,0,1,2\}$. Its expected value is zero and its variance is $(4+1+0+1+4)/5=2$. At $a=2$ the actual tail probability is $2/5$, while Chebyshev's inequality gives $2/2^2=1/2$. At $a=3$ the actual tail probability is zero, while the bound is $2/9$. A valid bound need not describe the exact shape or bounded support of a particular probability distribution.

For the standard normal distribution, the actual tail probabilities beyond two and three standard deviations are approximately $0.0455003$ and $0.0026998$, whereas the general bounds are $1/4$ and $1/9$. More knowledge of the probability distribution can therefore improve the estimate. The comparison does not diminish the guarantee: the three-point example in equality in Chebyshev's inequality attains it exactly.

The proof is a one-line use of Markov's inequality on the nonnegative random variable $Y=(X-\mu {)}^2$. The event $|X-\mu |\ge a$ equals $Y\ge a^2$, and $\mathbb{E}Y={\sigma }^2$. This explains why the denominator is squared and why a finite variance supplies the necessary information. ^[ref].

One important use is control of a sample mean. For independent and identically distributed random variables with finite variance ${\sigma }^2$, the variance of a sample mean is ${\sigma }^2/n$. HenceFor every fixed tolerance, the bound tends to zero as the sample size grows, proving the finite-variance case of the weak law of large numbers. For ${\sigma }^2=1$ and $\epsilon =0.1$, $n=2000$ makes the bound $0.05$. This is a sufficient bound under the stated assumptions, rather than an optimal required sample size.

An early historical source is Pafnuty Chebyshev's 1867 paper “Des valeurs moyennes”, published in French translation by N. de Khanikof. Its opening theorem bounds deviations of sums using their expected values and second moments, linking averages with control of exceptional outcomes. ^[ref].

The main modes include almost sure convergence, convergence in probability, convergence in Lp, and convergence in distribution. The strong law of large numbers and weak law of large numbers explain why a sample mean approaches an expected value; the central limit theorem describes the probability distribution of its remaining error after magnification by $\sqrt{n}$.

An independent and identically distributed sequence combines two conditions: its members are independent random variables, and every member has the same probability distribution. For a finite sequence $X_1,\dots ,X_n$ with common law $\nu$, this means that for all measurable sets $B_1,\dots ,B_n$,For an infinite sequence, require this condition for every finite collection of distinct indices. Equal marginal distributions supply the common law; independence supplies the product. Neither condition supplies the other. Grinstead and Snell call the combination an independent trials process.^[ref]

Choose a Bernoulli distribution with success probability $1/2$. Four independent draws produce all sixteen binary patterns with probability $1/16$ each. If instead $X_1=X_2=X_3=X_4=Z$ for one fair Bernoulli random variable $Z$, every member still has that same Bernoulli distribution, but only patterns 0000 and 1111 occur. Conversely, independently drawn Bernoulli random variables with success probabilities $1/4,3/4,1/4,3/4$ have a product joint probability distribution but unequal marginal distributions.

The combination is useful because one probability distribution describes each observation, while a product measure describes the full IID sample. In particular, an IID sample of finite-variance, real-valued observations has a predictable sample mean: its expected value stays at the population expected value, while its variance of a sample mean decreases as $1/n$. This is a mechanism behind Monte Carlo integration and the finite-variance weak law of large numbers. Whether real observations satisfy the independence and common-law assumptions is a separate modelling question.

Laws of large numbers give conditions under which sample means approach a deterministic expected value. For independent and identically distributed random variables with $\mathbb{E}|X_1|<\infty$, the strong law of large numbers gives almost sure convergence, while the weak law of large numbers gives convergence in probability. These conclusions specify different modes of convergence of random variables, rather than a universal numerical accuracy guarantee. The central limit theorem is a separate result about the remaining fluctuations after rescaling.

Pafnuty Chebyshev's 1867 paper “Des valeurs moyennes” develops bounds for deviations of sums in terms of their expected values and second moments. This connects his historical work with the modern Chebyshev's inequality and its use in the weak law of large numbers. ^[ref].

Let $\varphi (u)=\int e^{iuy}\mu (dy)$. Since $1-\cos (uy)\ge 0$, the Tonelli theorem giveswith the fraction defined to be one at zero. For $|x|\ge 1$, $\sin x/x\le \sin 1$. To verify the constant, the function is even; on $[1,\pi ]$ it decreases because $(x\cos x-\sin x{)}^{\prime }=-x\sin x<0$ on $(0,\pi )$ and $x\cos x-\sin x$ starts from zero. On $[\pi ,\infty )$, $\sin x/x\le 1/\pi <\sin 1$. Consequently the integrand is at least $1-\sin 1$ on $|y|\ge \lambda$, proving the characteristic-function tail boundFor pointwise convergence ${\varphi }_n\to \varphi$, where $\varphi$ is a characteristic function, the dominated convergence theorem applies on each finite integration interval, since the integrands lie between zero and two. HenceContinuity of $\varphi$ at zero and $\varphi (0)=1$ make the right side tend to zero as $\lambda \to \infty$. Given $\epsilon >0$, choose $\lambda$ so it is below $\epsilon /2$. The tail probability is then below $\epsilon$ for all sufficiently large $n$. Enlarge the compact interval to control the finitely many remaining measures, each of which has vanishing tails. This proves uniform tightness of the entire sequence, not just of its tail. It is the characteristic-function tightness bound principle with the sharper displayed constant.

For the integrable mean-zero summands, differentiation at zero is justified byThe dominated convergence theorem gives ${\varphi }^{\prime }(0)=i\mathbb{E}{X}_1=0$. DefineIt is continuous away from zero by continuity of the characteristic function, and continuous at zero by the zero derivative. Since $|\varphi (u)|\le 1$, the finite geometric series yields, for $n\ge 1$,and the case $n=0$ has both sides zero. Thus the requested uniform-in-$n$ estimate isThis is the characteristic-function derivative under an absolute first moment estimate.

Independence makes the characteristic function of $S_n/n=(X_1+\cdots +X_n)/n$ equal to $\varphi (u/n{)}^n$. Therefore $|1-\varphi (u/n{)}^n|\le |u|\psi (u/n)$. Applying the already proved characteristic-function tail bound to its distribution, for every $\epsilon >0$,The convergence follows because ${\sup }_{0\le u\le 1/\epsilon }\psi (u/n)\to 0$, and the remaining integral of $u$ is finite. Hence $\boxed{S_n/n\to 0\text{ in probability}}$, the mean-zero weak law of large numbers. This deduction uses the tail bound directly and does not presuppose the Lévy continuity theorem.

All three normalized sums converge in probability to $\mu$: the weak law of large numbers controls $S_L/L$, and the two fixed random variables vanish after division by $L$. Their rare fluctuations nevertheless differ. A fixed normal random variable would have to fluctuate by order $L$ to alter the mean by a fixed amount, and that costs $\exp (-\text{constant}{L}^2)$. A fixed exponential random variable can do the same for a cost of order $\exp (-\text{constant}L)$, exactly the large-deviation speed of the empirical mean.

Below $\mu +\lambda {\sigma }^2$, the optimal exponential contribution is zero, and the Gaussian empirical mean supplies the fluctuation. Above it, the optimal empirical mean remains at $\mu +\lambda {\sigma }^2$, while $B/L$ supplies $x-\mu -\lambda {\sigma }^2$. This explains the quadratic-to-linear change in the rate function and why only the upper tail is altered. Convergence to the same typical value does not imply the same large-deviation behavior.

The Schumacher compression theorem concerns a memoryless quantum information source with average density matrix $\rho$ on a finite-dimensional Hilbert space. For $n$ uses the average state is ${\rho }_n={\rho }^{\otimes n}$. Compression consists of an encoding quantum channel into a space of dimension $D_n$ and a decoding channel back to the original block space, with asymptotic rate $n^{-1}{\log }_2{D}_n$ qubits per source use. The optimal reliable rate is $S(\rho )$: every larger rate is achievable, and no smaller rate is reliable. This holds for average pure-source fidelity; the direct construction also preserves a purification reference, and the converse for that stronger entanglement fidelity criterion holds with fidelity tending to zero below the threshold. The encoding and decoding are deterministic channels; no free classical transmission of the source label is allowed.

Diagonalize $\rho ={\sum }_j{\lambda }_j|j⟩⟨j|$ and omit zero-probability eigenvalues when sampling. Define the quantum typical subspace projector $P_{n,\delta }$ by retaining product eigenvectors withThe sampled random variable $-{\log }_2{\lambda }_j$ has expectation $S=S(\rho )$. The weak law of large numbers gives $w_n=\mathrm{Tr}({\rho }_n{P}_{n,\delta })\to 1$. Every retained eigenvalue is at least $2^{-n(S+\delta )}$, and the total probability is at most one, soThis proves the two typical subspace theorem estimates used in the direct part.

Write $P=P_{n,\delta }$. Encode its range isometrically into a compressed space and reserve one orthogonal failure flag. Decode the range back isometrically and the failure flag to a fixed pure state $|\phi ⟩$. This typical-subspace compression with a failure flag is a physical channel whose composite action isIndeed its Kraus operators are $P$ and $|\phi ⟩⟨e_a|$, where the $e_a$ form an orthonormal basis of $\ker P$, and their adjoint products sum to $I$. The Kraus formula for entanglement fidelity givesFor any pure-state ensemble of ${\rho }_n$, the average squared quantum fidelity is at least this number: apply convexity of $|z{|}^2$ separately to the amplitudes $⟨{\psi }_x|K_{\alpha }|{\psi }_x⟩$ with their source probabilities. Thus average fidelity also tends to one. The compressed dimension is at most $2^{n(S+\delta )}+1$, and its rate is at most $S+\delta +o(1)$. Choose $\delta <R-S$ for any prescribed $R>S$. One may also take ${\delta }_n=n^{-1/4}$: the finite spectral information variance and Chebyshev's inequality still give $w_n\to 1$, with rate tending to $S$. A pure source, $S=0$, can simply be re-prepared exactly without sending any qubits.

For the coherent converse, we prove the finite-dimensional quantum compression converse. If the compressed space has dimension $D$, the state of reference plus compressed system has a pure-state decomposition with Schmidt rank at most $D$. Applying decoder Kraus operators locally cannot increase the Schmidt rank of those vectors, so the final reference–output state has Schmidt number at most $D$. Let $|{\Psi }_{{\rho }_n}⟩$ purify the source, with squared Schmidt coefficients equal to the decreasing eigenvalues ${\mu }_1\ge {\mu }_2\ge \cdots$ of ${\rho }_n$. For any normalized vector $|v⟩$ of Schmidt rank at most $D$, its reference support lies in a rank-at-most-$D$ projector $Q$. HenceThe last bound follows by maximizing ${\sum }_j{\mu }_j⟨j|Q|j⟩$ subject to $0\le ⟨j|Q|j⟩\le 1$ and their sum at most $D$. Average over a Schmidt-rank-bounded decomposition of the final state to obtain $F_e\le {\sum }_{j=1}^D{\mu }_j$.

At rate $R<S$, choose $0<\delta <S-R$. The total source probability outside the quantum typical subspace tends to zero, while every eigenvalue inside it is at most $2^{-n(S-\delta )}$. Therefore for $D_n\le 2^{nR}$,This argument allows arbitrary encoding and decoding channels, not only subspace projection.

For completeness, average pure-source reliability also forces the same rate, even though it is a weaker criterion than entanglement fidelity. Here is a separate converse for that formulation. Let the emitted block pure states be ${\psi }_x$, with probabilities $p_x$, and let ${\sigma }_x$ be the decoded states. If their mean squared fidelity tends to one, then their mean trace distance ${\bar{\delta }}_n$ tends to zero, since $\frac{1}{2}\Vert {\sigma }_x-{\psi }_x{\Vert }_1\le \sqrt{1-⟨{\psi }_x|{\sigma }_x|{\psi }_x⟩}$. This inequality follows by choosing a purification with that overlap and using contraction of trace distance under a partial trace.

We need only an elementary entropy continuity from Jordan decomposition bound, derivable from Question 4. If $\delta =\frac{1}{2}\Vert \sigma -\tau {\Vert }_1$, write the positive and negative parts of $\sigma -\tau$ as $\delta {\omega }_{+},\delta {\omega }_{-}$. The common density matrixand the two mixture bounds giveHere $g$ is increasing and concave, $g(0)=0$. Apply this in block dimension $d^n$, both to the average states and to individual outputs, using convexity of trace distance and concavity of $g$. Since the original ensemble is pure, its Holevo quantity is $S({\rho }_n)=nS$. The decoded ensemble hasThe Holevo quantity under a quantum channel cannot increase. To specify the entropy ingredient, Strong subadditivity of Von Neumann entropy states $S(XB)+S(BE)\ge S(B)+S(XBE)$. Attach the classical source label $X$, dilate the decoder isometrically to $BE$, and trace out $E$. The displayed inequality says $I(X:B)\le I(X:BE)$; the isometry preserves the input mutual information. Since $I(X:B)$ of a labelled ensemble is its Holevo quantity, this proves the needed decoder monotonicity. Before decoding, the system has dimension $D_n$, so its Holevo quantity is at most ${\log }_2{D}_n$. Thus ${\log }_2{D}_n\ge nS-o(n)$ and the rate cannot be below $S$. Both operational fidelity formulations therefore have the claimed noiseless coding threshold.

Normalization uses ${\int }_{-\infty }^{\infty }(1+x^2{)}^{-1}dx=\pi$, henceEach variable therefore has the standard Cauchy distribution. Let $S_n=X_1+\cdots +X_n$. We prove by induction that its probability density function isIt holds for $n=1$. If it holds for $n=m$, independence and convolution of probability densities giveThe supplied integral identity makes this $(m+1)/\{\pi [(m+1{)}^2+x^2]\}$, completing the induction. Thus addition increases the Cauchy scale to $n$.

For ${\overline{X}}_n=S_n/n$, the density transformation is $f_{{\overline{X}}_n}(x)=n{f}_{S_n}(nx)$, givingThe Cauchy sample mean has an unchanged distribution. In particular, for every $\delta >0$,independently of $n$, so the average does not converge in probability to zero.

There is no contradiction with the usual weak law of large numbers, whose finite-first-moment hypothesis is absent: $\mathbb{E}|X_1|=(2/\pi ){\int }_0^{\infty }x/(1+x^2)dx=\infty$. The positive and negative parts of $X_1$ both have infinite expectations, so $\mathbb{E}{X}_1$ is undefined. Its symmetric principal value is zero, but a principal value is not an expected value. The law therefore supplies no conclusion that these sample averages must approach zero.

To correct every pattern of $t_N=\lfloor pN\rfloor$ errors by nearest-codeword decoding, it suffices to have minimum Hamming distance ${\delta }_N=2t_N+1$: the corresponding radius-$t_N$ Hamming balls are disjoint. For $0<p<1/4$, the relative distance tends to $2p<1/2$, so the asymptotic Gilbert–Varshamov bound guarantees code rate at least $1-h_2(2p)$ in the limit. Therefore its immediate strict-rate guarantee isThe binary entropy is strictly increasing from zero to one on $[0,1/2]$, since $h_2^{\prime }(x)={\log }_2((1-x)/x)>0$ on $(0,1/2)$. Let $a=h_2^{-1}(1-R)$ denote the inverse on this interval. The asymptotic condition becomes

There is also a finite-length boundary refinement for the precise requirement of correcting $\lfloor pN\rfloor$ errors. If $d/N=\theta \le 1/2$, then each weight ${\theta }^j(1-\theta {)}^{N-j}$ for $j\le d$ is at least ${\theta }^d(1-\theta {)}^{N-d}$. Expanding $(\theta +(1-\theta ){)}^N$ gives the entropy bound for a Hamming ballThe case $d=0$ follows directly from $v_N(0)=1$. Taking $d=2t_N$ and using monotonicity of the binary entropy, the Gilbert–Varshamov bound yieldsThus the worst-case correction guarantee from the Gilbert–Varshamov bound even includes $p=a/2$ for the exact radius $\lfloor pN\rfloor$, with the number of messages rounded to an integer when necessary. No positive-rate guarantee of this form is obtained for $p\ge 1/4$; one must not extend the increasing-entropy calculation past $2p=1/2$. Failure of this sufficient bound does not rule out better codes.

By contrast, the Shannon second coding theorem gives vanishing error probability on the binary symmetric channel wheneverThe factor of two comes from demanding disjoint correction balls for every error pattern of a prescribed weight. This is a worst-case distance requirement, and the Gilbert–Varshamov bound is itself only a sufficient existence bound. The Shannon second coding theorem controls the probability of decoding failure for the random channel; rare bad patterns may remain.

Finally, the number of actual errors has binomial distribution with mean $pN$. Correcting exactly its mean does not itself imply error probability tending to zero. To get that conclusion from the distance construction when $R<1-h_2(2p)$, choose $\epsilon >0$ with $p+\epsilon <1/4$ and $R<1-h_2(2(p+\epsilon ))$, and correct $\lfloor (p+\epsilon )N\rfloor$ errors. The weak law of large numbers then makes the probability of exceeding the correction radius tend to zero.

Write $u(Y_i;\theta )={\partial }_{\theta }\log f(Y_i;\theta )$ and $U_n(\theta )={\sum }_{i}u(Y_i;\theta )$ for the individual and total score functions. Under the usual regularity assumptions, ${\mathbb{E}}_{{\theta }_0}u=0$ and ${\mathrm{Cov}}_{{\theta }_0}(u)=I({\theta }_0)$, the nonsingular per-observation Fisher information matrix. The central limit theorem and weak law of large numbers giveRearranging the stipulated score expansion, including its remainder, givesSlutsky theorem therefore proves the asymptotic normality of a maximum likelihood estimator:A consistent plug-in Fisher information matrix yields the Wald statistic $W_n=n({\hat{\theta }}_n-{\theta }_0{)}^{T}I({\hat{\theta }}_n)({\hat{\theta }}_n-{\theta }_0)$. Under the null hypothesis, $W_n\stackrel{d}{\to }{\chi }_d^2$, so a test of asymptotic size $\alpha$ rejects when $W_n$ exceeds the $(1-\alpha )$ quantile of that chi-squared distribution. A consistent Observed Fisher information matrix gives the same limiting test.

With an $r$-dimensional parameter of interest $\psi$ and nuisance parameter $\lambda$, partition $I$ accordingly. The marginal asymptotic covariance matrix of $\hat{\psi }$ is the $\psi \psi$ block of $I^{-1}$, whose inverse is the efficient informationThus the nuisance-adjusted Wald statistic is $W_{\psi ,n}=n(\hat{\psi }-{\psi }_0{)}^T{I}_{\mathrm{eff}}(\hat{\theta })(\hat{\psi }-{\psi }_0)$, with limiting null distribution ${\chi }_r^2$. Using $I_{\psi \psi }$ alone is generally incorrect when the nuisance parameter is estimated; it is valid when the cross-information vanishes.

For the Inverse Gaussian distribution in this problem, expand the exponent to obtain the log likelihood, up to terms independent of both parameters,Its score functions areFor each positive $\psi$, the first score changes sign from positive to negative at $\lambda =\bar{Y}$, so this is the global maximizing mean parameter of an exponential family. Substituting it in the second score givesThe reciprocal function is strictly convex, so the denominator is positive for a nonconstant sample. The profile log likelihood is strictly concave in $\psi$ and tends to minus infinity at both endpoints, proving this is a maximum. Under continuous model sampling, a sample of size at least two is nonconstant almost surely. For a constant sample, including a one-observation sample, no finite joint maximum-likelihood estimator exists: the likelihood increases without bound as $\psi \to \infty$ at $\lambda =\bar{Y}$.

For one observation, differentiating again and using $\mathbb{E}Y=\lambda$ givesThis parameter orthogonality proves that estimating $\lambda$ does not reduce the efficient information for $\psi$. The Wald statistic has the same algebraic form in the two cases:Using information evaluated at ${\psi }_0$ instead is another first-order equivalent convention.

The printed claim of coincidence needs a first-order interpretation. Known and unknown $\lambda$ need not give numerically identical Wald statistics, because their maximum-likelihood estimators differ. With $\lambda$ known, direct maximization givesFor example, observations $1,2$ and known $\lambda =1$ give ${\hat{\psi }}_K=4$ and ${\hat{\psi }}_U=12$. At ${\psi }_0=1$, the displayed Wald convention gives $9/16$ and $121/144$, respectively. Thus exact equality is false. Under the null hypothesis, however, $\bar{Y}-\lambda =O_p(n^{-1/2})$, hence ${\hat{\psi }}_K-{\hat{\psi }}_U=O_p(n^{-1})$. Their centered, standardized estimates differ by $o_p(1)$, so their Wald statistics differ by $o_p(1)$. This is the valid asymptotic coincidence supplied by inverse Gaussian shape estimation and nuisance orthogonality.

Use base-two logarithms and put $H=H(U_1)=-{\sum }_i{p}_i{\log }_2{p}_i$, with $0{\log }_{2}0=0$. Zero-probability letters can be omitted. The random variables $I_j=-{\log }_2{p}_{U_j}$ are self-information values, are IID, and have finite mean $H$ because the alphabet is finite. Independence givesFor an arbitrary $\delta >0$, define the typical setThe weak law of large numbers implies $P(T_{n,\delta })\to 1$. Every string in this set satisfiesThis is the asymptotic equipartition property for the given finite IID source, here obtained directly from the weak law of large numbers.

For the upper bound, summing the lower bound on typical-string probabilities yields $|T_{n,\delta }|\le 2^{n(H+\delta )}$. For sufficiently large $n$, its probability is at least $1-\epsilon$, so it is an admissible set in the definition of $M(n,\epsilon )$. HenceFor the lower bound, let $A$ be any admissible set and write ${\eta }_n=P(T_{n,\delta }^c)\to 0$. ThenEach member of this intersection has probability at most $2^{-n(H-\delta )}$. Therefore, for all sufficiently large $n$,This applies in particular to a minimum-size admissible set, giving a liminf at least $H-\delta$. Letting $\delta \downarrow 0$ proves both existence and the value of the minimal high-probability source-set exponent:For a deterministic source, $H=0$ and $M(n,\epsilon )=1$, consistent with the formula.

For coding, label the strings in an admissible set with distinct binary indices of length $\lceil {\log }_{2}M(n,\epsilon )\rceil$. A decoder reconstructs those strings exactly and may fail outside the set. Thus asymptotically $H$ bits per source symbol describe any fixed fraction $1-\epsilon$ of the probability mass. More strongly, every rate $R>H$ permits block codes with error tending to zero: choose $\delta <R-H$, encode the typical strings individually and reserve one index for all other strings. The required number of indices is at most $2^{n(H+\delta )}+1\le 2^{nR}$ for large $n$.

Conversely, a block encoder with at most $2^{nR}$ outputs can reconstruct at most that many distinct input strings correctly. If $R<H$, choose $0<\delta <H-R$. Its success probability is at mostThis gives the strong converse for fixed-rate source coding. It is the operational significance of the information entropy limit for reliable source encoding at a rate. The allowance of a small reconstruction error matters: a fixed-length code that must encode every possible string without error generally needs the larger rate ${\log }_2|\{i:p_i>0\}|$.

For fixed $\epsilon >0$, let the quantum typical subspace ${\mathcal{T}}_{\epsilon }^{(n)}$ be spanned by the product eigenvectors with nonzero eigenvalues satisfyingEquivalently their spectral information $-n^{-1}{\log }_2{\lambda }_{\mathbf{j}}$ lies within $\epsilon$ of $S(\pi )$. Denote its orthogonal projection by $P_{\epsilon }^{(n)}$.

The typical subspace theorem states that for every $\delta >0$ there is $n_0$ such that for $n\ge n_0$,Also the operator bounds on its range areTo see the concentration statement, sample $j_s$ independently with probabilities ${\lambda }_j$. The mean of $-{\log }_2{\lambda }_{j_s}$ is $S(\pi )$, so the weak law of large numbers gives typical probability tending to one. Zero-eigenvalue letters have probability zero and are excluded. The dimension bounds follow by summing the upper and lower typical eigenvalue bounds and using the typical mass between $1-\delta$ and $1$.

Here is a tripartite Schmidt projection entanglement concentration protocol. For $0<p<1$, set $q=1-p$. Regrouping the $n$ local qubits of each party giveswhere $|x|$ counts ones. Alice performs a projective measurement of that count, without learning the individual string. The orthogonal projector for outcome $k$ is ${\Pi }_k={\sum }_{|x|=k}|x⟩⟨x|$. By the Born rule, its probability and conditional pure state areAlice sends $k$ by classical communication; all three parties use local unitary operations to relabel the same ordered type class as $0,\dots ,d_k-1$. There are now $d_k$ equal common-label amplitudes. Measuring the individual qubits instead would have destroyed precisely this coherence.

To obtain an integer number $f$ of target GHZ states, choose $m=2^f\le d_k$. The following exact LOCC step handles type classes whose sizes are not powers of two. For every $m$-element subset $S\subseteq \{0,\dots ,d_k-1\}$, Alice uses the Kraus operatorEvery label occurs in $\left(\genfrac{}{}{0ex}{}{d_k-1}{m-1}\right)$ such subsets, proving ${\sum }_S{K}_S^{†}{K}_S=I$ on the occupied subspace. Thus this is a complete measurement in quantum mechanics. Each outcome has probability $m/[d_k(\genfrac{}{}{0ex}{}{d_k-1}{m-1})]=1/\left(\genfrac{}{}{0ex}{}{d_k}{m}\right)$, and leaves the normalized pure state $m^{-1/2}{\sum }_{r\in S}|r,r,r⟩$. After Alice communicates $S$, all parties relabel its elements in the same order. Binary expansion of the labels then factors the state into exactly $f$ independent copies of the target GHZ state. Local extensions of these operations handle unused basis vectors and do not affect the protocol.

For example, taking $f(k)=\lfloor {\log }_2{d}_k\rfloor$ gives an outcome-dependent yield without any failure branch. Since $K$ has a binomial distribution with parameters $(n,q)$, the weak law of large numbers gives $K/n\to q$ in probability. Stirling's formula, uniformly when $k/n$ stays away from zero and one, givesBy continuity of the binary entropy, $f(K)/n\to h_2(q)=h_2(p)$ in probability. The integer rounding and logarithmic correction have no first-order effect.

A fixed yield with success probability tending to one is also explicit. For sufficiently large $n$, takeOn $|K-nq|\le n^{2/3}$, the bounded derivative of the binary entropy in a neighborhood of the fixed $q\in (0,1)$ and Stirling's formula give ${\log }_2{d}_K\ge n{h}_2(q)-C_q{n}^{2/3}-O(\log n)\ge f_n$. Chebyshev's inequality bounds the complementary probability by $q(1-q)n^{-1/3}$. Abort there if necessary, and otherwise use the subset measurement in quantum mechanics with $m=2^{f_n}$. ThusThis matches the first-order rate in part (i), with exact outputs on every successful forward branch. If $p=1/2$, simply retain the original $n$ target copies; if $p$ is an endpoint, output zero target copies.