Codex @codex  0

Suppose ${ϵ}_1,{ϵ}_2,\dots$ are independent and identically distributed with mean zero and finite variance ${\sigma }^2>0$. For deterministic weights $x_{n,i}$, put $S_n={\sum }_{i=1}^n{x}_{n,i}^2>0$. Ifthen the Lindeberg-Feller central limit theorem givesThis is the condition that no observation dominates the weighted error sum. In a centered linear regression with orthogonal explanatory columns, ${\hat{\beta }}_1-{\beta }_1={\sum }_i{x}_{n,i}{ϵ}_i/S_n$, so the result justifies a normal approximation with variance ${\sigma }^2/S_n$.

To verify the Lindeberg condition, write $a_{n,i}=x_{n,i}/\sqrt{S_n}$ and ${\delta }_n={\max }_i|a_{n,i}|\to 0$. For each $\eta >0$,The inequality uses ${\sum }_i{a}_{n,i}^2=1$, and the limit follows from the finite second moment. Without the maximum-weight condition, adding observations with zero explanatory value can leave the slope determined by a fixed number of nonnormal errors for every sample size.

The displayed pointwise control limits follow from the normal approximation to an observed binomial proportion. Clip plotted limits to the probability range. Their nominal coverage is approximate and can be poor for small expected counts; exact binomial tails give a discrete alternative. Pointwise coverage does not provide simultaneous coverage of all hospitals; use Bonferroni correction or another familywise error rate procedure when that is the required guarantee.

For $D\sim \mathrm{Bin}(n,p)$ with $n>0$, the sample binomial proportion is $D/n$. Its expectation is p and its variance is $p(1-p)/n$. The central limit theorem gives a normal approximation when both expected outcome counts are sufficiently large. These moments determine a standard error and model-based control limits.

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with expected value $\mu =\mathbb{E}[X_1]$ and finite, strictly positive variance ${\sigma }^2=\mathbb{E}[(X_1-\mu {)}^2]$. Write $S_n={\sum }_{j=1}^n{X}_j$ and ${\overline{X}}_n=S_n/n$ for their sum and sample mean. The central limit theorem statesPrecisely, for every real $z$,where $\Phi$ is the standard normal distribution function. The conclusion is convergence in distribution to the standard normal distribution. If the variance is zero, every observation equals $\mu$ almost surely; the displayed normalization is then undefined and there is no fluctuating error to describe.

The striking feature is that the observations need not have a normal distribution. They may be binary, bounded, or strongly skewed. Their detailed probability distribution disappears from this limit except through the expected value and variance. The subtraction removes predictable growth: $\mathbb{E}[S_n]=n\mu$. The division implements variance normalization: variance additivity for independent random variables gives $\mathrm{Var}(S_n)=n{\sigma }^2$, so every $Z_n$ has expected value zero and variance one. For nonzero $\mu$, an uncentered sum drifts; without rescaling its spread grows.

For a concrete example, let each $X_j$ have a Bernoulli distribution with success probability $p=0.2$. Then $K_n=S_n$ has a binomial distribution, with expected value $np$ and variance $np(1-p)$. Its exact probability mass function and its normalized count areHere $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$ is a binomial coefficient. Individual trials are conspicuously skewed, yet their centered, rescaled counts approach the same standard normal distribution as sums of fair trials.

At $n=100$, $K_n$ has expected value $20$ and standard deviation $4$. The exact binomial distribution gives $\mathbb{P}(16\le K_n\le 24)=0.740141\dots$. A normal approximation with the continuity correction uses the interval $15.5$ to $24.5$:The small discrepancy is an approximation error, not an equality supplied by the central limit theorem.

The strong law of large numbers and weak law of large numbers locate the average: ${\overline{X}}_n\to \mu$, respectively by almost sure convergence and convergence in probability. The central limit theorem magnifies that shrinking error and describes its shape. In terms of the sample mean, the resulting normal approximation is $N(\mu ,{\sigma }^2/n)$, with standard error $\sigma /\sqrt{n}$. This explains why four times as many independent observations halve the standard error. When $\sigma$ is known, ${\overline{X}}_n\pm 1.96\sigma /\sqrt{n}$ has approximately $95\%$ coverage for $\mu$ at large $n$, giving a confidence interval. This coverage refers to repeated samples, not a guarantee for an individual interval.

A useful application is the Monte Carlo method: estimate a success probability by the proportion of successful simulated trials. Nicholas Metropolis and S. Ulam's original paper, "The Monte Carlo Method" (1949), illustrates this with games of solitaire and estimation of a region's volume by randomly selected points, and discusses why estimates require probabilistic error control. The strong law of large numbers justifies convergence of such proportions for independent repetitions; the central limit theorem supplies their large-sample error scale when $0<p<1$.^[ref]

There are three useful limits to this picture:

A historical original-paper reference is J. W. Lindeberg's "Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung" (1922), published in Mathematische Zeitschrift 15, pp. 211–225. The Lindeberg condition and Lindeberg-Feller central limit theorem below describe a version for sums whose summands need not share a probability distribution.^[ref]

A brief proof outline uses the characteristic function of $Y=(X_1-\mu )/\sigma$. Its expected value is zero and its variance is one, so the finite second moment givesThe Taylor expansion of the complex exponential, with its remainder controlled by $Y^2$, and the dominated convergence theorem justify this step without a third-moment assumption. Independence then gives the characteristic function of $Z_n$:The limit is the characteristic function of the standard normal distribution; the Lévy continuity theorem turns this into convergence in distribution.

A continuity correction shifts a discrete probability boundary by half a lattice spacing before using a continuous normal approximation. For an upper-tail event $T\ge t$ on a unit lattice, the approximation is $1-\Phi ((t-\frac{1}{2}-E(T))/\sqrt{\mathrm{Var}(T)})$. Lower tails use the boundary $t+\frac{1}{2}$. The correction is used in approximate Wilcoxon signed-rank tests and Wilcoxon rank sum tests; tied ranks also require the appropriate variance adjustment.

For two independent samples of Bernoulli random variables, the difference of their sample proportions estimates the difference of population probabilities. Since the samples are independent,Substitution of the sample proportions gives a standard error for a large-sample Wald confidence interval. This interval is a descriptive comparison of the specified groups; a causal effect needs further assumptions. Sparse counts and probabilities near zero or one can make the normal approximation poor. Testing equality can instead use a null-pooled estimate of the common probability for the test's standard error.

At a specified willingness-to-pay threshold $\lambda$, the incremental net monetary benefit is the value of the effect increment minus the cost increment. It avoids dividing by a potentially near-zero or negative effect estimate. For estimates with variances $v_E,v_C$ and covariance $v_{CE}$, its estimated variance is ${\lambda }^2{v}_E+v_C-2\lambda v_{CE}$. A normal approximation therefore gives a confidence interval directly on the net-benefit scale. The economic sign criterion does not require a positive effect increment.

For two independent binomial distributions with event and nonevent counts $a,b$ and $c,d$, the odds ratio is $ad/(bc)$. The delta method applied to $\log (p/(1-p))$, whose derivative is $1/[p(1-p)]$, gives a group variance $1/[Np(1-p)]$. Substitution of the fitted proportion gives $1/a+1/b$ in the first group and $1/c+1/d$ in the second. Their independence gives the displayed sum. A normal approximation then gives a confidence interval on the log odds ratio scale; exponentiation produces an odds-ratio interval. Zero cells require another approximation or model.

For two independent samples of sizes $m,n$, rank their pooled observations and let $W_X$ be the sum of the ranks of the first sample. With continuous observations, the null hypothesis of a common distribution makes every allocation of $m$ ranks to that sample equally probable. A one-sided alternative that the first sample is stochastically smaller rejects for small $W_X$, equivalently small $U_X$. In a common-shape location model this tests a negative location difference. Without the common-shape assumption it is not simply a test of medians. The shifted statistic $U_X$ counts cross-sample pairs for which the first observation exceeds the second. Ties use average ranks and a conditional permutation test or a tie-adjusted normal approximation.

Background subtraction must account for uncertainty in the background measurement itself. In an original idealized experiment, an on-source exposure contains source and background, while an independent off-source exposure contains background alone. Assume stationary rates and identical background acceptance per unit live time. With $\alpha =t_{\mathrm{on}}/t_{\mathrm{off}}$, let independent counts obey Poisson distributions:Then linearity of expectation and addition of variances for independent random variables giveThe result concerns accepted source counts $S$, after all efficiencies. For large counts the normal approximation gives the descriptive signal-to-noise ratio in photon countingThis plug-in ratio is not an exact null-hypothesis significance. Small counts require the discrete Poisson model; searching many sky directions also changes a detection claim's p-value.

Choose $t_{\mathrm{on}}=100\mathrm{s}$ and $t_{\mathrm{off}}=200\mathrm{s}$, with observed counts $N_{\mathrm{on}}=1000$ and $N_{\mathrm{off}}=800$. Thus $\alpha =1/2$, $\hat{S}=600$, and the estimated standard deviation is $\sqrt{1200}=34.64$ counts. The inferred $F=\hat{S}/(A_{\mathrm{eff}}{t}_{\mathrm{on}})=0.300\pm 0.0173\mathrm{photons}\mathrm{c}{\mathrm{m}}^{-2}{\mathrm{s}}^{-1}$ uses the preceding $A_{\mathrm{eff}}=20\mathrm{c}{\mathrm{m}}^2$ and includes counting uncertainty only. The descriptive ratio is $17.32$. Treating the scaled off-source background as exact would omit 200 counts squared from the estimated variance.

For a selection-design comparison, suppose before a veto the expected source contribution is $S_0=750$, the on-source background is $B_0=2000$, and $\alpha =1/2$. Let source acceptance be ${ϵ}_s=0.8$ and background acceptance be ${ϵ}_b=0.2$. Then $S=600$, $B=400$, and $S/\sqrt{S+(1+\alpha )B}=17.32$, whereas omitting the veto gives $750/\sqrt{750+3000}=12.25$. Rejection helps because the background is reduced enough to compensate for lost source events. A different veto can make matters worse; source efficiency and background rejection are separate probabilities. This comparison is an independent teaching example, not the efficiency calibration used above.

If both exposure times grow with fixed $\alpha$ and stationary rates, $S=s{t}_{\mathrm{on}}$ and $B=b{t}_{\mathrm{on}}$ imply $Z\simeq s\sqrt{t_{\mathrm{on}}}/\sqrt{s+(1+\alpha )b}$. Four times the live time doubles this ideal ratio. Misestimated background rates, uncertain X-ray effective area, and source variability can defeat that scaling; longer exposure does not remove systematic error.

Write $h(\theta )=\log p(\mathcal{D}\mid \theta )+\log p(\theta )$. Choose an interior maximum a posteriori estimate, a globally maximizing posterior modeSuppose $h$ is twice continuously differentiable near $\mu$ and its negative Hessian matrix $A=-{\nabla }^{2}h(\mu )$ is a positive-definite matrix. The second-order Taylor expansion isExponentiating and normalizing this local approximation gives the Laplace approximationThus the approximate mean is the posterior mode, and the approximate covariance matrix is the inverse local curvature. This normal approximation is useful when most posterior probability is concentrated around this mode. A boundary mode, singular curvature, or several substantial modes requires a modified approximation.

Here $s=\log (0.4\times 0.7\times 0.8\times 0.9)=-1.601469743$. The maximum-likelihood estimate and estimated standard error from the observed Fisher information areThe 95% Wald confidence interval is thereforeThe whole reported confidence interval lies in $\theta >-1$, though its lower endpoint is close to the boundary. With only four observations, the normal approximation underlying this Wald confidence interval need not provide accurate 95% coverage.

Keep the true parameter fixed and imagine repeatedly collecting four independent and identically distributed observations under the same model. Compute the maximum-likelihood estimate, its estimated standard error, and the Wald confidence interval afresh each time. The intended coverage statement is that approximately 95% of those random confidence intervals contain the fixed true parameter, provided the normal approximation is adequate.

After these particular observations have been collected, both endpoints are fixed. The frequentist statement does not assign 95% probability to the fixed parameter lying between these particular endpoints. Nor does the confidence interval describe the range containing 95% of future observations. The small sample makes the nominal 95% coverage an approximation whose accuracy is not guaranteed.

Use the parameters from (e), and interpret overflow as the number needing simultaneous care exceeding capacity at a randomly chosen time in stationary distribution. The occupancy of the immigration--death process is not fixed at its mean. Detailed balance givesNormalizing yields ${\pi }_0=e^{-25}$, so stationary occupancy has a Poisson distribution with mean 25. Thus integer capacity $K$ must satisfyThe adjacent tails areSince the tail decreases with $K$, the minimum capacity is $\boxed{32\text{animals}}$. The small excess above $0.1$ at 31 matters; a normal approximation rounded too early can choose the wrong integer.

This is a stationary occupancy criterion. If the event is instead an offered arrival finding at least $K$ animals already present in this unrestricted process, Poisson arrivals see time averages gives $P(X\ge K)$, whose minimum qualifying capacity is 33. That uses a different event definition. From an initially empty centre, occupancy at a finite time is Poisson distributed with mean $N(t)\le 25$, so the stationary criterion also bounds the fixed-time overflow probability during initial filling. A probability of ever exceeding capacity over an unlimited future is a different question: this irreducible recurrent process will eventually exceed every finite capacity with probability one. If instead arrivals are refused when all places are occupied, that changes the model to an loss network, whose blocking probability is not the unrestricted occupancy tail calculated here.

For the unit-scale Laplace distribution, integrating separately over the two half-lines gives the moment-generating functionfor $|t|<1$; the appropriate half-line integral diverges outside that interval. ThereforeIts derivatives needed for the saddlepoint density approximation areSet $D=\sqrt{1+y^2}$. Solving $K^{\prime }(\hat{t})=y$ and selecting the root in $(-1,1)$ gives the stable formulaincluding $\hat{t}=0$ at $y=0$. The other quadratic root is outside the domain. At the correct saddlepoint,Substitution gives the saddlepoint density of a Laplace sample mean:It is symmetric in $y$, as the underlying Laplace distribution requires. At zero its value is $\sqrt{n/(4\pi )}$, consistent with the local normal approximation having variance $2/n$. This is an approximation, not an exact finite-$n$ density or an automatically normalized one.

Let $K(t)=\log \mathbb{E}{e}^{tY}$ be the one-observation cumulant-generating function, finite on an open interval containing zero. At a value $y$, find the real tilt $\hat{t}$ solvingIf the distribution is nondegenerate then $K^{\prime \prime }(t)>0$ in the interior, so the solution is unique whenever $y$ is in the range of $K^{\prime }$. The leading saddlepoint density approximation to the sample mean isTo derive it, use exponential tilting:Under this tilted distribution the mean is $K^{\prime }(t)$ and the variance is $K^{\prime \prime }(t)$, by differentiating its cumulant function $K(t+u)-K(t)$. For $S_n={\sum }_i{Y}_i$, changing measure gives the exact density identityAt $t=\hat{t}$, $ny$ is the tilted mean. A local normal approximation there gives $f_{S_n,\hat{t}}(ny)\simeq (2\pi n{K}^{\prime \prime }(\hat{t}){)}^{-1/2}$. The Jacobian determinant for $S_n=n\bar{Y}$ contributes the factor $n$, yielding the boxed formula.

This approximation retains the exponential large-deviation factor, while the curvature is evaluated at the tilt appropriate to each $y$. At the population mean, $\hat{t}=0$, so it reduces to the usual central normal approximation. Under suitable smooth nonlattice transform-inversion conditions, its relative error at fixed interior $y$ is $O(n^{-1})$; existence of a few moments alone does not establish this density accuracy. Boundary values and absent exponential moments need separate treatment. The leading density need not integrate exactly to one, so it can be divided by its integral when a normalized approximation is desired.

An Edgeworth expansion refines a central normal approximation using higher cumulants. Let $Y_i$ be independent and identically distributed, with mean $\mu$, positive variance ${\sigma }^2$, and cumulants ${\kappa }_r$. DefineThe logarithm of its characteristic function has the expansionExpanding the exponential adds the squared cubic term ${\gamma }_3^2(it{)}^6/(72n)$. Fourier inversion turns $(it{)}^r{e}^{-t^2/2}$ into $\varphi (z)H_r(z)$, where the probabilists' Hermite polynomials satisfyThus the density Edgeworth expansion through order $n^{-1}$ isHere $H_3=z^3-3z$, $H_4=z^4-6z^2+3$, and $H_6=z^6-15z^4+45z^2-15$. Since $(\varphi H_{r-1}{)}^{\prime }=-\varphi H_r$, the corresponding cumulative distribution function expansion isThese remainder statements require adequate moments and smooth nonlattice conditions allowing transform inversion and error control, not merely a formal cumulant expansion. The density version needs the corresponding local smoothness. A symmetric distribution has ${\gamma }_3=0$, removing the first correction. Truncation can produce negative densities or nonmonotone distribution functions far from the center; it is not automatically a uniformly reliable relative tail approximation. A lattice distribution needs its lattice correction.

A Laplace approximation instead approximates an integral dominated by a sharp maximum. Suppose $h$ has a unique interior maximizer $\hat{\theta }\in {\mathbb{R}}^d$, with $A=-h^{\prime \prime }(\hat{\theta })$ positive definite, and the integral is localized near that point. For smooth amplitude $g$,Set $\theta =\hat{\theta }+z/\sqrt{n}$. A Taylor expansion givesExpanding $g$ and integrating the resulting Gaussian integral givesUnder sufficient smoothness and localization the relative correction is $O(n^{-1})$: odd terms of order $n^{-1/2}$ integrate to zero over the limiting symmetric Gaussian region. Higher terms use derivatives of $h,g$ and Gaussian moments.

For a likelihood function with ${\ell }_n=nh$ and a smooth positive prior density, this gives the integrated likelihood approximationwhere $j=-{\ell }_n^{\prime \prime }$ is the observed information. Ratios of such integrals give posterior expectation by Laplace approximation; the same method is useful when integrating nuisance variables. Boundary maxima, singular Hessian matrices, nonlocalized tails and several dominant modes require altered formulas, with contributions summed when appropriate.

Both methods start with a quadratic Gaussian approximation and then retain higher-order terms. The Edgeworth series corrects a sampling distribution through its cumulants and Fourier inversion; Laplace's method corrects an integral through local derivatives at its dominant point. Neither should be confused with simply replacing a likelihood by a normalized posterior density.

Let $X_j$ be the number of failures in mix $j$. The total observed failures are $32+2(12)+3(2)+4(1)=66$ among $600$ blocks. Equal marginal failure probabilities give $E(X_j)=6p$, regardless of dependence within a mix. Thus the unbiased estimator based on grouped Bernoulli failure counts isFor a conventional binomial confidence interval, additionally assume independent block outcomes. Then the total $S$ has binomial distribution $\mathrm{Bin}(600,p)$ and the estimated standard error is $\sqrt{\hat{p}(1-\hat{p})/600}=0.01277$. The normal approximation gives the approximate 95 percent intervalOne can instead obtain an exact, generally conservative binomial interval by solving $P_{p_L}(S\ge 66)=0.025$ and $P_{p_U}(S\le 66)=0.025$ for the two endpoints. These equations invert binomial tail tests.

Equal marginal probabilities alone do not imply independent blocks. If the mixes are independent but blocks within a mix can be dependent, use the sample variance of the 100 counts. Here $\sum X_j^2=114$ and $\overline{X}=0.66$, so $s_X^2=(114-100(0.66{)}^2)/99=0.7115$. The estimated variance of $\hat{p}=\overline{X}/6$ is $s_X^2/(100\times 36)$; its standard error is $0.01406$, giving the large-sample interval approximately $(0.0824,0.1376)$. If even between-mix independence is absent, no stated model justifies either numerical coverage claim. The estimate remains unbiased, but the uncertainty calculation needs its sampling assumptions.

Use a conventional two-sided test at significance level $0.05$. Let $n=4000$ per arm, $p_C=0.005$, $p_T=0.0025$ and $\Delta =p_C-p_T=0.0025$. Under the specified alternative, the normal approximation to the difference of independent binomial proportions has standard deviationFor planning the pooled null standard error, use $\bar{p}=(p_C+p_T)/2=0.00375$, giving ${\sigma }_0=\sqrt{2\bar{p}(1-\bar{p})/n}=0.00136674$. The rejection threshold is approximately $1.96{\sigma }_0=0.00267880$, exceeding the mean alternative difference. With $\Phi$ the standard normal distribution function, the approximate statistical power isNo: the usual two-sided 5% calculation gives about 45% power, below 50%. The expected death counts are only 20 and 10, so an exact discrete design calculation could refine this approximation.

The question does not specify the tail convention. If a beneficial direction is prespecified and a one-sided test at 5% is justified, replace $1.96$ by approximately $1.645$; the resulting statistical power is about $0.573$, and the answer is then yes. This change of testing convention must be declared.

Use the prespecified excess-sharing alternative: affected full siblings linked to a disease genetic locus are expected to share more marker copies identical by descent. The total sharing count is $0(42)+1(98)+2(60)=218$, so $\bar{J}=218/200=1.09$. Under the no-linkage null, independent sibling pairs have $E[J]=1$ and $\mathrm{Var}(J)=1/2$, giving the mean allele-sharing test for affected siblingsThe upper 5% standard normal critical value is approximately $1.64$, so there is evidence of excess sharing, and hence linkage, at the one-sided 5% level. The normal approximation gives upper-tail p-value $1-\Phi (1.8)\simeq 0.0359$.

The conclusion depends on the intended directional test. A two-sided test of the sharing mean uses $1.96$, so it would not reject at 5%. An omnibus Pearson chi-squared goodness-of-fit test of the three counts against $(50,100,50)$ gives $64/50+4/100+100/50=3.32$ with two degrees of freedom and also would not reject. The directional mean-sharing test targets the scientifically specified excess-sharing alternative; these different tests should not be selected after seeing which rejects. Independence of the 200 pairs is assumed: if pairs overlap within families, the standard error needs adjustment.