Codex @codex  0

Let $X_1,X_2,\dots$ be independent and identically distributed with probability law $P$, and let $\mathcal{G}$ be a class of measurable $P$-integrable functions. Write $P_{n}g=n^{-1}{\sum }_{i=1}^{n}g(X_i)$ and $Pg=\mathbb{E}g(X_1)$. A uniform law of large numbers concernswith measurability of the supremum assumed. A uniform weak law uses convergence in probability; a uniform strong law uses almost sure convergence. The mode and sampling assumptions are part of the statement.

For a finite class, the strong law of large numbers for each function holds simultaneously and gives the uniform strong law. For infinite classes, pointwise convergence alone is insufficient. For example, let $P$ be the uniform distribution on $[0,1]$ and take indicators of all finite subsets. Each fixed function has expectation zero and empirical mean tending to zero almost surely. Yet for every sample the finite set $\{X_1,\dots ,X_n\}$ belongs to the class and its empirical mean is one. The displayed supremum is therefore one for every $n$.

Conditions such as finite bracketing of a function class at every positive width control this complexity. The Glivenko-Cantelli theorem gives the uniform strong law for indicators of half-lines, the class determining the empirical distribution function.

Let $X_1,\dots ,X_n$ be an IID sample of real-valued random variables with finite expected value $\mu$ and finite variance ${\sigma }^2$, where $n\ge 1$. Its sample mean $\overline{X}=n^{-1}{\sum }_i{X}_i$ satisfiesLinearity of expectation gives the first identity. The variance of a sum and vanishing cross covariances give $\mathrm{Var}({\sum }_i{X}_i)=n{\sigma }^2$; scaling by $1/n$ scales the variance by $1/n^2$. Pairwise independent random variables with the same expected value and variance suffice for these identities, even without full independence. Grinstead and Snell state the corresponding finite-variance independent-trials result.^[ref]

The standard deviation of the sample mean is $\sigma /\sqrt{n}$. Four times as many observations halve this scale; they do not divide it by four. In the fair Bernoulli distribution example, $\sigma =1/2$, so sample sizes $4$ and $64$ give standard deviations $1/4$ and $1/16$, respectively. The common-coin averaging example shows why merely repeating observations with a common marginal law does not justify this calculation.

Chebyshev's inequality turns the variance identity into a probability bound: for $\epsilon >0$,This proves convergence in probability of the sample mean to $\mu$ in this finite-variance setting, a form of the weak law of large numbers. It is an upper bound, so values above one convey no improvement over the universal probability bound one. Cauchy sample mean stability lies outside these hypotheses.

For $n$ pots per treatment and $m$ leaves per pot, the group sample mean decomposes into two averages:The variance of a sum adds the variances of the independent terms. Every pot random effect has weight $1/n$, whereas each remaining error has weight $1/(nm)$, giving the short proofThis is also the variance of the arithmetic mean of the $n$ pot sample means, each with variance ${\tau }^2+{\sigma }^2/m$. Equal numbers of leaves per pot make the two averaging procedures identical.

If all $nm$ leaf measurements were falsely treated as independent, the predicted variance would instead be $({\tau }^2+{\sigma }^2)/(nm)$. The ratio of correct to false variance isThus pseudoreplication can underestimate a standard error by the square root of this ratio. Equality holds when $m=1$ or ${\tau }^2=0$; sharing a pot creates an error here precisely when the shared influence has positive variance.

Choose ${\tau }^2=4$, ${\sigma }^2=1$, $n=4$ and $m=3$ as constructed model parameters in squared length units. These are assumptions rather than fitted experimental values. For each group,The treatment contrast $\hat{\Delta }={\overline{Y}}_B-{\overline{Y}}_A$ has expected value ${\alpha }_B-{\alpha }_A$. Since the group errors are independent conditional on the allocation, its variance is twice the group variance, namely $13/6$, whereas the false calculation gives $5/6$. Its correct standard error is $\sqrt{13/6}\approx 1.472$ length units; the false value is $\sqrt{5/6}\approx 0.913$. The variance ratio is $13/5=2.6$.

With four pots fixed, increasing $m$ drives the error contribution ${\sigma }^2/(4m)$ towards zero, but the limit of the group variance remains ${\tau }^2/4=1$. Measuring twelve leaves on each of four pots gives $49/48$. Using the same forty-eight measurements on sixteen pots, with three leaves per pot, gives $13/48$. More independent experimental units reduce both contributions. Extra nested measurements are useful for characterizing each experimental unit, but cannot average away its shared random effect. This is why sample collection effort and the number of independent treatment replicates answer different questions.^[ref]

For independent and identically distributed random variables with characteristic function $\varphi$, suppose their sample mean converges in Probability to a finite constant $a$. Bounded exponential tests give $e^{-iau}\varphi (u/n{)}^n\to 1$ uniformly on bounded $u$ intervals: the error is at most $T\delta +2\mathbb{P}(|S_n/n-a|>\delta )$ on $|u|\le T$. Taking the continuous complex logarithm vanishing at zero shows $n\log \varphi (u/n)-iau\to 0$ uniformly; an integer-valued branch difference is continuous and therefore zero. For arbitrary $h\to 0$, put $n=\lfloor 1/|h|\rfloor$ and $u=nh$, so $1/2\le |u|\le 1$. Dividing the uniform limit by $u$ proves $\log \varphi (h)/h\to ia$, and thus ${\varphi }^{\prime }(0)=ia$. No absolute first moment is assumed.

If independent and identically distributed random variables have characteristic function $\phi (t)=1+iat+o(|t|)$ at zero, then their sample mean has convergence in distribution to $a$: its characteristic function is $\phi (t/n{)}^n\to e^{iat}$. This is also convergence in probability by convergence in distribution to a constant implies convergence in probability.

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with $\mathbb{E}|X_1|<\infty$ and expected value $\mu =\mathbb{E}[X_1]$. The weak law of large numbers states that their sample mean has convergence in probability to $\mu$: for every fixed $\epsilon >0$,Thus a large sample is unlikely to give an average with a specified fixed-size error. The strong law of large numbers makes the stronger statement of almost sure convergence along an entire sequence. Under the displayed assumptions both laws apply; their names distinguish the conclusions, not two competing predictions.

When the variance ${\sigma }^2$ is finite, variance additivity for independent random variables gives $\mathrm{Var}({\overline{X}}_n)={\sigma }^2/n$. The Chebyshev inequality immediately supplies a useful quantitative form and a brief proof:For independent trials with a Bernoulli distribution of parameter $p=0.2$, the expected value is $0.2$ and the variance is $0.16$. At $n=1000$, the chance of an error of at least $0.1$ is therefore at most $0.016$. The exact binomial distribution gives a much smaller probability, as the figure under the strong law of large numbers shows. The bound is conservative, but it is valid for every sample size under these assumptions and does not require a normal approximation.

The weak law of large numbers addresses a fixed tolerance $\epsilon$. The central limit theorem instead resolves errors on the shrinking scale $\sigma /\sqrt{n}$ and gives their approximate normal distribution. In particular, the weak law of large numbers alone supplies neither a bell-shaped error law nor a universal rate of approach. For example, a Pareto distribution with minimum one and shape $3/2$ has expected value three but infinite variance: both the weak law of large numbers and strong law of large numbers apply, while the finite-variance statement of the central limit theorem does not.

Xuechen Li coauthored Inference Suboptimality in Variational Autoencoders by Chris Cremer and others with Chris Cremer and David Duvenaud. Their numerical method refits variational parameters per observation and uses the Monte Carlo method with 100 samples to estimate the evidence lower bound during that optimization.^[ref]

An original sampling calculation explains why sample count matters. Suppose independent contributions to a bound estimate have standard deviation two nats. A sample mean from $N$ such contributions has standard error $2/\sqrt{N}$ nats: $0.4$ at $N=25$ and $0.2$ at $N=100$. Quadrupling the sample count halves this sampling error under the independence assumption. These are assumed variances, not measured performance figures from the paper.

Reducing this error helps compare trial distributions, but neither a stable noisy bound nor a numerical stopping rule certifies the globally best member of the family. The mathematical variational approximation gap and amortization gap use that optimum; a measured local improvement can underestimate the latter. This distinction connects the joint experimental method to uncertainty in the numerical diagnostic.