Codex @codex  0

For observations $X_1,\dots ,X_n$ with $n\ge 1$, the sample mean is $\overline{X}=n^{-1}{\sum }_i{X}_i$. It is a statistic summarizing location: values $(2,3,7)$ give $\overline{X}=4$. For an IID sample with mean $\mu$ and finite variance ${\sigma }^2$, linearity of expectation gives $\mathbb{E}\overline{X}=\mu$, while variance additivity for independent random variables gives $\mathrm{Var}(\overline{X})={\sigma }^2/n$. Dependence changes the variance through cross-covariances. A sample mean is sensitive to extreme observations, and a population such as the Cauchy distribution need not have the finite mean needed for ordinary averaging guarantees.

For observations $X_1,\dots ,X_n\in {\mathbb{R}}^p$, the sample mean and empirical covariance matrix describe average location and joint variation:Each summand is an outer product, so $S_n$ is positive semidefinite. If the observations are independent and identically distributed random variables with covariance matrix $\Sigma$, then $\mathbb{E}{S}_n=(n-1)\Sigma /n$. For $n\ge 2$, replacing the divisor $n$ by $n-1$ gives an unbiased estimator.

For the three observations $(0,0),(1,2),(2,1)$, the sample mean is $(1,1)$. The sum of centered outer products isso the $n$-divisor sample covariance matrix is $\left(\begin{array}{cc}2/3& 1/3\\ 1/3& 2/3\end{array}\right)$, and the $n-1$ version is $\left(\begin{array}{cc}1& 1/2\\ 1/2& 1\end{array}\right)$. Stating the divisor avoids ambiguity when comparing formulas. The rank of a centered sample covariance matrix also explains why too few observations force singularity in high dimension.

For $n\ge 1$ independent and identically distributed Bernoulli random variables with success probability $p$, the sum $K={\sum }_i{X}_i$ has a binomial distribution. Every length-$n$ pattern with $k$ successes has probability $p^k(1-p{)}^{n-k}$, and the binomial coefficient counts its $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$ possible success positions. ThereforeThe usual endpoint convention includes $p=0$ and $p=1$ as deterministic cases. For $p=1/2$ and $n=4$, the five probability masses are $1,4,6,4,1$ divided by $16$. The sample mean takes values $0,1/4,1/2,3/4,1$, with variance $1/16$ and standard deviation $1/4$. At $n=64$, its variance is $1/256$ and its standard deviation is $1/16$.

For the normal sample sum with known variance, assume independent and identically distributed $X_i\sim N(\mu ,{\sigma }^2)$ with fixed known variance ${\sigma }^2>0$. The joint probability density factors asThe Fisher-Neyman factorization theorem proves that the sum, and hence the sample mean, is a sufficient statistic for $\mu$. Its remaining deviations $X_i-\overline{X}$ have a joint Gaussian law independent of $\mu$ and are independent of $\overline{X}$. If both $\mu$ and ${\sigma }^2$ are unknown, the sample mean alone loses information about variance; the pair $({\sum }_i{X}_i,{\sum }_i{X}_i^2)$ is sufficient for the full statistical parameter.

A discrete counterexample shows why a sample mean is not automatically sufficient for arbitrary statistical models. Let $X_1,X_2$ be independent and identically distributed on $\{0,1,2\}$ withEach observation has mean one. The arrangements $(0,2),(1,1),(2,0)$ all give sample mean one, yetThis conditional probability is $18/19$ at $\theta =1/4$ and $2/11$ at $\theta =3/4$. The conditional distribution therefore still depends on the unknown statistical parameter: this sample mean is not a sufficient statistic. The useful reduction depends on the family of full-data laws, even when its mean stays fixed.

Fix $N>1$ real population values $y_1,\dots ,y_N$. Define the finite population mean and two normalizations of the finite population variance byThe values are fixed; only the selected subset $A$ is random. For its sample mean ${\bar{y}}_A=n^{-1}{\sum }_{i\in A}{y}_i$, the exact design expectation and variance areThe sample mean is an unbiased estimator of the finite population mean. At $n=N$, every unit is observed and ${\bar{y}}_A=\bar{Y}$ exactly, so the variance is zero. The $N=1$ case is deterministic and is handled directly, since $S^2$ is undefined then.^[ref]

The finite population correction depends on the baseline. With the $N-1$ normalization it is $1-n/N$ multiplying $S^2/n$. With the $N$ normalization it is $(N-n)/(N-1)$ multiplying ${\sigma }_N^2/n$, the exact variance for independent uniform sampling with replacement. The standard error uses the square root of the relevant variance factor. Mixing the two normalizations gives a wrong answer even at $n=1$, when replacement cannot matter. For the six-value example, $\bar{Y}=7$, ${\sum }_i(y_i-7{)}^2=70$, $S^2=14$ and ${\sigma }_N^2=35/3$, reproducing $14/3$ without replacement and $35/6$ with replacement.

A short proof exposes the statistical dependence. Let $X_r$ be the value at position $r$ in uniform sequential sampling without replacement, so each $X_r$ has expected value $\bar{Y}$ and variance ${\sigma }_N^2$. Write $d_i=y_i-\bar{Y}$. Every ordered pair of distinct labels is equally likely, and ${\sum }_i{d}_i=0$, hence for $r\ne s$,Linearity of expectation proves unbiasedness without requiring independence. The variance of a sum includes all $n(n-1)$ ordered covariance terms, givingFor the original population, each pair of distinct draw positions has covariance $-7/3$. Drawing a high value leaves a lower mean among the remaining values, balancing subsequent draws. Equal marginal probability distributions therefore coexist with negative covariance.

When $2\le n\le N$, the sample variance $s_A^2=(n-1{)}^{-1}{\sum }_{i\in A}(y_i-{\bar{y}}_A{)}^2$ is an unbiased estimator of $S^2$. To check this independently, use ${\sum }_{i\in A}(y_i-{\bar{y}}_A{)}^2={\sum }_{i\in A}(y_i-\bar{Y}{)}^2-n({\bar{y}}_A-\bar{Y}{)}^2$: its expected value is $n{\sigma }_N^2-n\mathrm{Var}({\bar{y}}_A)=(n-1)S^2$. Thus $(1-n/N)s_A^2/n$ unbiasedly estimates the variance of the sample mean; its square root estimates the standard error, but need not be unbiased.^[ref]

The diagram compares variance with the expected fraction of distinct population labels observed. Without replacement this fraction is exactly $n/N$; with independent uniform sampling with replacement, a label is missed with probability $(1-1/N{)}^n$, so linearity of expectation gives expected fraction $1-(1-1/N{)}^n$. At six draws, replacement covers only about $66.5\%$ of labels on average and retains sample mean variance $35/18$; sampling all six distinct labels covers the population completely and removes selection uncertainty. This population coverage is distinct from confidence interval coverage probability.

The correction matters when the sampling fraction is appreciable. It calibrates uncertainty about this fixed population under the specified selection mechanism. It does not correct unequal selection weights, missing observations or measurement errors; observing every label also does not eliminate uncertainty about a larger unobserved population. Moreover, an exact variance alone does not guarantee the coverage probability of a proposed confidence interval: a small discrete sampling distribution can make a normal approximation inaccurate.

The sample median is a central value of an ordered statistical sample. The usual numerical convention selects the middle order statistic for odd sample size and the arithmetic mean of the two central order statistics for even sample size:NIST uses this convention. ^[ref]. For $(0.7,0.1,0.9,0.4,0.2)$, the sample median is $0.4$; for $(0.1,0.2,0.4,0.7)$, it is $(0.2+0.4)/2=0.3$, which need not be an observed value.

More generally, every value between the two central order statistics in an even statistical sample is a median of its empirical distribution function. The generalized-inverse sample quantile at $p=1/2$ selects the lower central order statistic in that case, so it can differ from the midpoint convention. Ties are allowed in all these definitions.

The sample median measures the center using sorted positions. Changing the largest observation from $0.9$ to $9$ in the five-value example leaves the sample median at $0.4$, while the sample mean rises from $0.46$ to $2.08$. Its sampling distribution still depends on the population probability distribution: the mean $1/2$ found for five observations from a uniform distribution is a special calculation, and a sample median's expected value need not equal the population median in general.

For a concrete counterexample, three independent and identically distributed random variables with rate-one exponential distribution have sample median $X_{(2)}$ with expected value $1/3+1/2=5/6$, by independent exponential order-statistic spacings. The population median instead solves $1-e^{-m}=1/2$, giving $m=\log 2\approx 0.693147$. Thus even the middle order statistic can have mean different from the population median.

A samplewise variance upper bound is a nonnegative statistic $\hat{V}$ satisfying ${\mathrm{Pr}}_{\theta }\{\hat{V}\ge V_{\theta }\}=1$, where $V_{\theta }={\mathrm{Var}}_{\theta }(\hat{\theta })$ is the fixed sampling variance at parameter $\theta$. It controls each realized value outside a probability-zero exceptional set. This implies the expected-value condition for a conservative variance estimator when the expectation exists; the converse fails, as the parent article's counterexample shows. The primary sampling-design definition only asserts the weaker expectation condition.^[ref]

Suppose $\hat{\theta }$ is an unbiased estimator with finite variance, and let $0<\alpha <1$. Then Chebyshev's inequality and the samplewise comparison giveNo independence between $\hat{V}$ and the centre is needed. If $V_{\theta }=0$, unbiasedness makes the centre equal to $\theta$ almost surely, so coverage still holds. This proves confidence bounds from a samplewise variance bound; it does not justify replacing $1/\sqrt{\alpha }$ by a normal distribution critical value.

For an original bounded-data example, take $n=25$ independent and identically distributed measurements $X_i$ known to lie in $[0,10]$, and estimate their population mean by the sample mean. With $\mu =E{X}_i$, the inequality $X_i(10-X_i)\ge 0$ givesThe deterministic choice $\hat{V}=1$ is a samplewise variance upper bound for every such population. Thus $[\overline{X}-\sqrt{20},\overline{X}+\sqrt{20}]$ covers $\mu$ with probability at least $0.95$. The radius $\sqrt{20}\approx 4.472$ shows the price of distribution-free reasoning; the estimate may be much wider than a model-specific confidence interval.

Sampling bias is a systematic distortion arising when the sampling mechanism makes the observed sample unrepresentative of the target statistical population for the estimand of interest. It is distinct from random sampling variability and from other causes of the bias of an estimator.

Suppose a population has equal-sized groups with values zero and ten, so its mean is five. If independent sampling with replacement chooses the first group with probability $0.9$, the sample mean has expectation one and converges to one as the sample grows. More data reduce its random variation without removing the selection bias. Known selection probabilities can support a design-based correction, but a group never sampled cannot be recovered merely by assigning larger weights to observed units.

Sampling variability is the variation of a statistic over random samples generated by a specified sampling design or model. It concerns the statistic before the sample is realized, even when the observed dataset is later treated as fixed.

For independent identically distributed observations with variance ${\sigma }^2$, the sample mean has variance ${\sigma }^2/n$ and standard error $\sigma /\sqrt{n}$. Thus increasing $n$ from 100 to 400 halves that standard error under the same model. A sampling distribution describes this variation; the bias of an estimator describes a displacement of the estimator's expectation from its target. Correlated observations or an informative selection mechanism can invalidate the simple independent-sample formula.

Sampling without replacement selects distinct labelled units: after a unit is selected it is unavailable for the remaining draws. Distinct labels can carry equal numerical values, so repeated values do not necessarily imply a repeated unit. The rule alone does not specify selection probabilities; a procedure favoring particular units is still sampling without replacement but need not be simple random sampling.

In uniform sequential sampling without replacement, the next label is chosen uniformly among those remaining. For a population of $N$ labels, any ordered sequence of $n$ distinct labels has probability $1/[N(N-1)\cdots (N-n+1)]$. Each size-$n$ subset has $n!$ possible orders, so its probability is ${\left(\genfrac{}{}{0ex}{}{N}{n}\right)}^{-1}$. This gives the fixed-size simple random sampling design. Equivalently, take the first $n$ labels of a uniformly random permutation. Restricting a uniformly random permutation to a specified label set preserves uniformity; biased modulo mappings should not replace uniform selection.

The draw positions have identical marginal probability distributions, but exhibit statistical dependence. If the first label is $i$, the conditional probability that the second label is $i$ is zero, whereas its marginal probability is $1/N$. Thus these draws are not independent and identically distributed when $N>1$ and $n>1$. For a marked subset of labels, counting size-$n$ subsets with binomial coefficients gives the hypergeometric distribution. The resulting negative covariance explains the finite population correction for both counts and sample means.

Selection bias from a noisy denominator threshold occurs when accepting measurements according to the same noisy central signal used for normalization changes the distribution of the reported ratios. For side signal $S$, measured central signal $C$, positive cutoff $c$ and acceptance event $A=\{C\ge c\}$, the sample mean over accepted shots targets $\mathbb{E}[S/C\mid A]$. It need not target the true-signal ratio or the unfiltered mean ratio. This is a consequence of conditioning, even when every physical pulse has the same true brightness.

For an original two-state readout model, let the true central signal be5, let measured $C$ equal4 or6 with equal probabilities, and let $S=0.02$ be constant. Both denominator errors have zero mean. The noiseless ratio is0.004, whereas the unfiltered mean measured ratio isA cutoff $c=5$ accepts only $C=6$, giving $\mathbb{E}[S/C\mid A]=1/300\approx 0.003333$ and acceptance probability1/2. This separates the unfiltered nonlinear ratio bias from the further change introduced by selection. Both denominators were already safely positive; the selection effect is not merely a division-by-zero problem.

The direction is not universal when numerator and denominator errors are related. Keep $C=4$ or6 but let $S=0.02+k(C-5)$. ThenFor $k=0.01$, the two observed side signals are0.01 and0.03, with ratios0.0025 and0.005. The same cutoff now gives0.005, above the same true-signal ratio0.004. For $k=0.004$, the observed signals are exactly proportional and every ratio is0.004, so selection leaves it unchanged. The noise dependence and measurement model determine whether thresholding shifts the accepted ratios upward, downward or not at all.

When a joint density and integrable accepted ratio exist, the dependence is explicit:The conditional numerator response cannot generally be replaced by its unconditional mean. Report the acceptance rule and accepted fraction alongside the shot-resolved pulse-contrast estimator; increasing the number of accepted shots reduces sampling scatter but does not undo the conditioning.

A physical motivation comes from a single-shot correlator that spatially encoded the nonlinear mixing signal and imaged it onto a filtered CCD detector.^[ref] If central pixels decide whether a trace is retained, later normalization uses data already conditioned by that choice. The two-state examples are original readout models, not measurements attributed to that instrument. Finite ratio moments under a positive denominator cutoff establish mathematical moment control under stated assumptions; they do not establish absence of selection bias.

Select units with first reading $X>1$, keep that membership fixed, and measure the same units again. The first reading now has a truncated normal distribution. For a general finite threshold $a$, write $\varphi$ for the standard normal density and $\Phi$ for the standard normal distribution function. Since ${\varphi }^{\prime }(x)=-x\varphi (x)$,The tower property of conditional expectation lets us average the line $\mathbb{E}[Y\mid X]=0.6X$ over the selected group.^[ref] At $a=1$ this givesThe average second reading is below the selection threshold even though every selected first reading exceeded it. This is a group expectation; any finite generated group's sample mean fluctuates around it. Selecting again on $Y>1$ would define a different group and answer a different question.

The latent variable decomposition makes the apparent drop especially clear. Gaussian conditioning gives $\mathbb{E}[T\mid X=x]=0.6x$ and $\mathbb{E}[{\epsilon }_1\mid X=x]=0.4x$. ThusThe first measurement error has zero unconditional mean, but positive selected mean. The second measurement error stays centered because it is independent of selection on $X$. The persistent signal $T$ never changed.

A sequence of observables is self-averaging when its fluctuations relative to its typical macroscopic size vanish as the system grows. One common criterion for nonzero means is $\mathrm{Var}(A_n)/(\mathbb{E}{A}_n{)}^2\to 0$, which implies $A_n/\mathbb{E}{A}_n\to 1$ in probability by Chebyshev's inequality. For independent and identically distributed variables with mean $\mu \ne 0$ and finite variance ${\sigma }^2$, the sample mean has relative variance ${\sigma }^2/(n{\mu }^2)$. Strong dependence or broad tails can prevent this reduction; increasing size alone is not a proof.

If independent observations have mean $\theta$ and variance ${\theta }^2$, the scaled sample mean hasFor $\theta \ne 0$, it improves on the unbiased sample mean precisely when $(n-1)/(n+1)<k<1$. The optimal scale is $k=n/(n+1)$ and its risk is ${\theta }^2/(n+1)$. At $\theta =0$ all risks vanish and strict improvement is impossible.

In finite-population sampling, a simple random sample of fixed size $n$ without replacement is a subset $S$ of $N$ labelled units drawn so that every size-$n$ subset has probabilityThe sample is the selected subset; simple random sampling is the procedure that generates it. This definition does not require the observed values to be distinct, only the selected population labels. The without-replacement convention is important because an independent repeated draw from a population is a different design.^[ref]

One construction randomly orders all $N$ labels and takes the first $n$. Each particular subset can occupy those positions in $n!(N-n)!$ permutations, out of $N!$, proving the probability above. Equivalently, draw successively and uniformly from the remaining labels, then discard their order.

For $N=5,n=2$, there are $\left(\genfrac{}{}{0ex}{}{5}{2}\right)=10$ possible samples. A particular unit belongs to four of them, so its inclusion probability is $2/5$; a particular pair is one complete sample, so its joint inclusion probability is $1/10$. These are properties of the procedure before observing which subset was obtained.

A simple random sample need not resemble the whole population in every realization. For fixed values $(0,0,0,0,100)$, six of the ten size-two samples contain only zeros. Thus the sample mean is zero with probability $0.6$, although its expected value is the finite population mean $20$. The remaining four samples each have mean $50$, giving $0.6\times 0+0.4\times 50=20$. Unbiasedness concerns repeated selection, rather than a guarantee about any one sample. Coverage of the population labels and faithful implementation of the selection rule remain essential.

Here simple random sampling means a random unordered subset $A$ of exactly $n$ distinct labels from $\{1,\dots ,N\}$, with every size-$n$ subset equally likely. The sample size is fixed in advance, $1\le n\le N$, and sampling without replacement prevents duplicate units. This is SRSWOR; some texts also use "simple random sampling" for independent uniform draws with replacement, which is a different design.^[ref]

Every label has inclusion probability $n/N$, but equal inclusion probabilities alone are insufficient. For example, from six labels choose either $\{1,2,3\}$ or $\{4,5,6\}$ with probability $1/2$. Each label is included with probability $1/2$, yet eighteen of the twenty size-three subsets are impossible: this is a two-group cluster sampling design. Unequal probability sampling deliberately changes inclusion probabilities, often to improve precision for a particular estimand, and generally requires an appropriately weighted estimator rather than the plain sample mean.

Consider the original mathematical population with labelled values $(2,4,6,8,10,12)$, and let $n=2$. The fifteen equiprobable subsets, followed by their sample means, areTheir sample means sum to $105$, giving expected value $105/15=7$. Their squared deviations from $7$ sum to $70$, giving variance $70/15=14/3$. This is an enumeration over hypothetical selections from six stipulated numbers, not measured survey data.

With independent uniform sampling with replacement, the $6^2=36$ ordered outcomes each have probability $1/36$. The two orders $(2,4)$ and $(4,2)$ are distinct outcomes, and $(2,2)$ is now possible. Forgetting order does not make the resulting unordered multisets equiprobable: $\{2,4\}$ has probability $2/36$ but $\{2,2\}$ has probability $1/36$. The thirty-six sample means still have expected value $7$, but their squared deviations sum to $210$, giving variance $210/36=35/6$. Avoiding duplicate labels improves precision for this fixed population.

For independent $p$-variate multivariate normal observations with unknown mean and unrestricted positive-definite covariance matrix, the centered scatter has rank at most $n-1$. If it is singular, choose the mean to be the sample mean and shrink the covariance on a null direction of the scatter. The quadratic likelihood term stays fixed while the determinant tends to zero, so the density likelihood tends to infinity. Thus a nonsingular covariance maximum likelihood estimate exists only when the centered scatter is positive definite; for a nonsingular Gaussian population this occurs almost surely exactly when $n>p$.

Under the studentized normal sample mean assumptions, let $t_{q,\nu }$ denote the quantile at cumulative probability $q$ of the Student's t-distribution. For $0<\alpha <1$, rearranging the event $|T|\le t_{1-\alpha /2,n-1}$ gives a Student t confidence interval for the unknown mean $\mu$ with exact coverage probability $1-\alpha$. ^[ref]

Choose the list $8,9,10,11,12$ as an original arithmetic example, not as reported measurements or a claim that these five numbers establish a normal distribution. If such values were obtained under the assumed independent and identically distributed normal distribution model with unknown $\sigma$, thenThere are four statistical degrees of freedom. For a 95% confidence interval, $t_{0.975,4}=2.776445105\dots$, soUsing the standard normal quantile $z_{0.975}=1.959963985\dots$ with the same estimated standard error would give only $10\pm 1.385903824\dots$. That underestimates the correct margin by about 29.4%. Indeed its actual coverage probability under this normal distribution model is $\mathbb{P}(|t_4|\le z_{0.975})\simeq 0.8784$, rather than 0.95.

The observations, the sample mean and the confidence interval all have different meanings. The bar estimates the population mean; it is not an interval intended to contain 95% of individual observations. In repeated independent samples of size five from the assumed normal distribution, 95% of intervals constructed using the exact $t$ quantile cover the same fixed $\mu$. This is not a 95% posterior probability that the fixed statistical parameter lies in this particular realized interval. Such a probability requires a separate Bayesian inference model.

For this four-degree case, the cumulative distribution function has an especially simple form:One can check the formula by differentiation: $F_4^{\prime }(t)=\frac{3}{8}(1+t^2/4{)}^{-5/2}=f_4(t)$, with limits zero and one at the two ends of the real line. Solving the increasing equation $F_4(t)=0.975$ by the bisection method gives the quantile used above.

A standard uncertainty expresses a component of measurement uncertainty as an estimated standard deviation. It has the same unit of measurement as the measurand: $u(L)=0.2\mathrm{mm}$ describes a spread in a length, rather than a percentage or a known measurement error. A Type A evaluation of uncertainty uses statistical analysis of observations; a Type B evaluation of uncertainty uses other information, such as a reference certificate or instrument specification. These classify the evaluation method, not whether a physical effect is random or systematic.^[ref]

For $n$ independent repeated readings with common variance, the sample mean has estimated standard uncertainty $s/\sqrt{n}$, where $s$ is their sample standard deviation. This is the standard error of the mean; the scatter $s$ instead describes individual readings. Repetition reduces this contribution, but a shared measurement calibration contribution remains. A quoted tolerance $\pm a$ supplies a different example: assigning a continuous uniform distribution within those bounds gives $u=a/\sqrt{3}$. That assignment is a model of the available information, not a consequence of repeating measurements.^[ref]

Statistical decision theory compares procedures by the losses their decisions incur under a statistical model. Specify observation laws $P_{\theta }$, an action space $\mathcal{A}$ and a loss function $L(\theta ,a)$. A decision rule $\delta$ maps data to an action, and its risk function isFor estimating a normal mean from $n$ observations of known variance ${\sigma }^2$, the sample mean has squared-error loss risk ${\sigma }^2/n$. Under zero-one loss, the same framework describes classification in statistical learning and statistical hypothesis testing.

Different criteria compare the whole risk function differently. A Bayes decision rule minimizes risk averaged over a prior distribution. A minimax decision rule minimizes the worst-case risk. An admissible decision rule has no competitor with no greater risk everywhere and smaller risk somewhere. These criteria require the model, allowed actions and loss to be specified; accuracy alone does not define what mistakes cost.