Codex @codex  0

TODO: Complete the data-specific numerical results and diagnostics when the original Iris_data.csv is available.

The identity

\sum_{i} (x_{i} - μ)^{2} = \sum_{i} (x_{i} - \overset{x}{ˉ})^{2} + n (μ - \overset{x}{ˉ})^{2}

shows that the likelihood function is a bell-shaped curve in

μ

, maximized at the sample mean, with width

1/ n

. Its actual vertical scale also depends on the within-sample sum of squares.

Continue the R script with:

sse <- sum((x - xbar)^2)
loglik <- function(mu) {
  -n/2 * log(2*pi) - 0.5 * (sse + n * (mu - xbar)^2)
}
grid <- seq(3, 6, length.out = 2001L)
ell <- loglik(grid)
png("q2-likelihood.png", width = 900, height = 600)
par(mar = c(4.5, 4.5, 4, 1))
if (max(ell) > log(.Machine$double.xmin)) {
  plot(grid, exp(ell), type = "l", xlab = expression(mu),
       ylab = expression(L(mu)), main = "Likelihood on [3,6]")
} else {
  # Explicitly label a proportional likelihood if the true scale underflows.
  plot(grid, exp(ell - max(ell)), type = "l", xlab = expression(mu),
       ylab = "Likelihood relative to maximum on plotted grid",
       main = "Likelihood on [3,6] (rescaled)")
}
abline(v = xbar, col = "steelblue", lty = 2)
dev.off()

Working first with the log-likelihood avoids premature numerical underflow. The fallback preserves the shape on the requested interval and labels the altered scale. A data-specific plot requires the missing CSV file; no fabricated observation set is used here.

Past exam of the systems biology course of the University of Cambridge / 2024 / Paper 3 / 2 / b / i / Solution 2026-10-20

TODO: Complete the data-specific numerical results and diagnostics when the original Iris_data.csv is available.

The estimate requested is the sample mean, $\overset{x}{ˉ} = n^{- 1} i \sum x_{i}$ . The named electronic file Iris_data.csv is not present in the supplied source archive or this repository, and its measurements are not printed in the paper. Consequently its numerical sample mean cannot be recovered from the paper alone. The familiar built-in iris data must not be silently substituted: the question does not identify those observations as its sample.

The following R code reads the actual file and prints its sample mean. The later code blocks in Question 2 continue this same script and use the resulting x, n and xbar.

dat <- read.csv("Iris_data.csv", check.names = FALSE)
# Recognize common labels, or a single measurement column.
keys <- tolower(gsub("[^[:alnum:]]", "", names(dat)))
j <- which(keys %in% c("sepallength", "sepallengthcm", "x"))
if (length(j) == 0L && ncol(dat) == 1L) j <- 1L
if (length(j) != 1L) {
  stop("Select the sepal-length column explicitly after inspecting names(dat).")
}
x <- dat[[j]]
stopifnot(is.numeric(x), length(x) > 0L, all(is.finite(x)))
n <- length(x)
xbar <- mean(x)
cat("n =", n, " sample mean =", format(xbar, digits = 12), "\n")

This reader assumes a header row and one observation per row. An alternative layout should be identified explicitly rather than dropping measurements or treating an identifier column as a length.

Past exam of the systems biology course of the University of Cambridge / 2024 / Paper 3 / 2 / f / Solution 2026-10-20

TODO: Complete the data-specific numerical results and diagnostics when the original Iris_data.csv is available.

Estimate the posterior mean by averaging the retained states:

E [μ ∣ x] = \frac{1}{9000} t = 1001 \sum 10000 μ_{t} .

The exact truncated normal distribution reference is

E [μ ∣ x] = m + s \frac{ϕ ( m / s )}{Φ ( m / s )}, m = \overset{x}{ˉ} - \frac{1}{3 n}, s = n^{- 1/2} .

For a posterior well separated from zero, the truncation correction is tiny and the posterior mean is approximately

\overset{x}{ˉ} - 1/ (3 n)

, slightly below the sample mean.

The R continuation prints the Monte Carlo estimator, its approximate batch-based Monte Carlo standard error, and the exact value for comparison:

posterior_mean_mc <- mean(post)
posterior_mean_exact <- m + s * exp(dnorm(m/s, log = TRUE) -
                                    pnorm(m/s, log.p = TRUE))
# Nonoverlapping batches account approximately for serial dependence.
batch_length <- floor(sqrt(length(post)))
number_batches <- floor(length(post)/batch_length)
used <- post[seq_len(batch_length * number_batches)]
batch_averages <- colMeans(matrix(used, nrow = batch_length))
mcse <- sd(batch_averages)/sqrt(number_batches)
cat("Posterior mean, MCMC =", posterior_mean_mc,
    " approximate MCSE =", mcse, "\n")
cat("Posterior mean, exact =", posterior_mean_exact, "\n")

The batch length must be long enough relative to the chain's dependence for this uncertainty estimate to be reliable. No numerical data-specific posterior mean is determined by the supplied paper alone.

Principal component score 2026-10-14

A score is an observation's coordinate along a principal component direction. Center an observation by the fitted sample mean, apply the fitted variable scaling when using principal component analysis on a correlation matrix, then take its inner product with the unit principal component loading vector. New observations must use the training center and scaling. The score variance equals the corresponding eigenvalue when the covariance normalization is consistent.

Q-Q plot 2026-10-13

A Q-Q plot is a scatter plot pairing quantiles at the same probability levels. For a statistical sample of size

n

, sort the observations into order statistics

x_{(1)} \leq \dots \leq x_{(n)}

. Choose plotting positions

p_{i}

and a reference quantile function

Q_{0}

, then plot

(Q_{0} (p_{i}), x_{(i)}), p_{i} = \frac{i - 1/2}{n}, i = 1, \dots, n .

Here the horizontal coordinate is a reference value, and the vertical coordinate is an observed value; neither axis is the probability

p_{i}

. The displayed plotting positions are our chosen convention, placing every level strictly between zero and one. They are not a claim that the

i

th order statistic equals that population quantile. For two statistical samples, use sample quantiles at matching levels, with a stated interpolation convention when their sizes differ. ^[ref]

As a computed example, take

n = 19

and the standard normal quantiles

z_{i} = Φ^{- 1} (p_{i})

, where

Φ

is the standard normal cumulative distribution function. Construct the ordered values

a_{i} = 10 + 2 z_{i}

. At ranks

1, 10, 19

, the plotting positions are

1/38, 1/2, 37/38

, the reference quantiles are approximately

- 1.937932, 0, 1.937932

, and the constructed values are approximately

6.124137, 10, 13.875863

. Every point lies on

a = 10 + 2 z

by construction. These 19 values are deliberately chosen, not random draws or real measurements.

Figure 1.
An exact constructed Q-Q line encodes location and scale
. Left: all 19 pairs $(z_{i}, a_{i})$ lie on $a = 10 + 2 z$ , with plotting positions $p_{i} = (i - 1/2) /19$ . Right: subtracting the specified location 10 and dividing by the specified scale 2 puts the same points on the identity line. These parameters are construction constants, not sample mean and sample standard deviation estimates. The 19 values are deterministic examples, not random draws. **Details**.

The usefulness of a Q-Q plot is that it turns a comparison of probability distributions into geometry: if a reference random variable

Z

is transformed into

X = μ + σ Z

with

σ > 0

, its quantile function satisfies

Q_{X} (p) = μ + σ Q_{Z} (p)

. This follows because a positive scaling preserves ordering. Thus the location-scale family produces a straight population Q-Q plot, and an approximate sample line suggests a compatible shape over the observed range. Its intercept describes location and its slope describes scale. The identity line requires the same location, scale and units on both axes; even then its displayed angle is 45 degrees only with equal axis scales. Curvature is often more informative than a histogram because it shows which quantiles depart from the reference. ^[ref]

Rao-Blackwell estimator after interim selection Created 2026-10-06 Updated 2026-10-07

For two independent stage means

X, Y \sim N (δ, s^{2})

and continuation

X \geq c

, let

S = (X + Y) /2

and

v = s / 2

. The second-stage mean is conditionally unbiased. The Rao-Blackwell theorem gives

E (Y ∣ S, X \geq c) = S - v λ ((c - S) / v)

, using

X ∣ S \sim N (S, v^{2})

and truncation. This is a conditionally unbiased estimator using both stages through their combined sample mean.

Repeated-timing uncertainty in a pendulum gravity measurement 2026-10-18

For simple pendulum oscillation period

T

in the small-angle approximation,

g = 4 π^{2} L / T^{2}

. If the length measurement and

N

identically precise timing measurements have independent errors, the timing sample mean has standard error

σ_{T} / N

. Propagation of uncertainty gives the title formula. A target fractional standard uncertainty

ϵ

requires

N \geq 4 (σ_{T} / T)^{2} / [ϵ^{2} - (σ_{L} / L)^{2}]

, rounded upward to an integer. No amount of repeated timing can beat the floor

σ_{L} / L

without improving the length measurement; correlated errors or a timing systematic error require a different model.

Ridge regression Created 2026-09-24 Updated 2026-10-14

Ridge regression fits a linear regression model while penalizing the size of its slope regression coefficients. For a fixed

n \times p

design matrix

X

containing the slope predictors, observed responses

Y

, and an unpenalized regression intercept

α

, the objective is

(α_{λ}, β_{λ}) = α \in R, β \in R^{p} argmin {∥ Y - α 1 - Xβ ∥_{2}^{2} + λ ∥ β ∥_{2}^{2}}, λ \geq 0.

Here

1

is the all-ones vector, and

∥ β ∥_{2}^{2} = \sum_{j} β_{j}^{2}

is the squared Euclidean norm of the slopes alone. The first term is the residual sum of squares; the second implements regularization. No random-error assumption is needed to define this fit. The convention is a summed squared-error loss: writing the objective as

n^{- 1} RSS + λ_{avg} ∥ β ∥_{2}^{2}

gives the same fit when

λ = n λ_{avg}

.^[ref]

For an exact example, reuse the constructed pairs

(x_{i}, Y_{i}) = (0, 1), (1, 2), (2, 2), (3, 4), (4, 5)

from linear regression. Their sample means are

\overset{x}{ˉ} = 2

and

\overset{ˉ}{Y} = 14/5

, and the centered cross-products are

\sum_{i} (x_{i} - 2)^{2} = 10

and

\sum_{i} (x_{i} - 2) (Y_{i} - 14/5) = 10

. Thus

β_{λ} = \frac{10}{10 + λ}, α_{λ} = \frac{14}{5} - 2 β_{λ}, Y_{i} = \frac{14}{5} + β_{λ} (x_{i} - 2) .

The ordinary least squares line has slope

1

and residual sum of squares

4/5

. At

λ = 10

, ridge regression has slope

1/2

, regression intercept

9/5

, and residual sum of squares

33/10

; at

λ = 30

these become

1/4

23/10

, and

257/40

. Every line passes through

(2, 14/5)

: the penalty reduces the slope while retaining the response sample mean.

Figure 1.
Exact synthetic ridge fits flatten around the sample mean as training RSS increases
. **Details**.

The computational benefit is visible after predictor centering. Write

X_{c}

for the design matrix with each column's sample mean subtracted, and

Y_{c} = Y - \overset{ˉ}{Y} 1

. The unpenalized intercept in ridge regression gives

α_{λ} = \overset{ˉ}{Y} - \overset{x}{ˉ}^{T} β_{λ}

, and the closed-form ridge regression estimator solves

(X_{c}^{T} X_{c} + λ I_{p}) β_{λ} = X_{c}^{T} Y_{c} .

For any nonzero vector

v

and

λ > 0

v^{T} (X_{c}^{T} X_{c} + λ I_{p}) v = ∥ X_{c} v ∥_{2}^{2} + λ ∥ v ∥_{2}^{2} > 0

. This positive-definite matrix is therefore invertible even when

X_{c}

lacks full column rank, including repeated columns or

p \geq n

. The unique penalized slopes resolve the ambiguity of rank-deficient ordinary least squares; they do not supply evidence about slope directions absent from the data. This makes ridge regression useful for multicollinearity.^[ref]

Predictor centering and scaling serve different purposes. Centering handles the free regression intercept. Scaling determines what counts as a large slope: replacing

x_{j}

c x_{j}

changes its coefficient to

β_{j} / c

for the same predictions, changing its contribution to an unweighted penalty. A common convention forms standard scores

z_{ij} = (x_{ij} - \overset{x}{ˉ}_{j}) / s_{j}

using each nonconstant predictor's sample standard deviation

s_{j}

, then penalizes slopes

θ_{j}

in those coordinates. Since

θ_{j} = s_{j} β_{j}

, this is the weighted penalty

λ \sum_{j} s_{j}^{2} β_{j}^{2}

in the original units. Constant columns have

s_{j} = 0

and must be removed before this scaling. Scaling is a modelling choice; equal penalties in original units and in standardized units define different fits.^[ref]

The singular value decomposition explains the shrinkage. If

X_{c} = U D V^{T}

is the compact decomposition using the

r

positive singular values

d_{j}

, with columns

u_{j}

U

, then

Y_{λ} = \overset{ˉ}{Y} 1 + \sum_{j = 1}^{r} \frac{d _{j}^{2}}{d _{j}^{2} + λ} u_{j} u_{j}^{T} Y_{c} .

(4)

The principal-component shrinkage by ridge regression factors are between

0

and

1

: directions with small singular values are damped most. As

λ \to \infty

, the fitted values approach

\overset{ˉ}{Y} 1

; the vanishing-penalty ridge limit recovers the ordinary least squares fitted vector as

λ ↓ 0

, even if the slope estimates at zero penalty are nonunique. The covariance and bias of a ridge regression estimator describes the resulting bias-variance tradeoff. A penalty can reduce variance of an estimator while introducing bias of an estimator; the unbounded directional risk of fixed ridge shrinkage rules out a uniform improvement for a fixed positive penalty over all possible signals.^[ref]

For a second exact construction, choose a centered orthonormal set of three vectors,

u_{1} = \frac{1}{2} (1, - 1, 1, - 1)^{T}, u_{2} = \frac{1}{2} (1, 1, - 1, - 1)^{T}, u_{3} = \frac{1}{2} (1, - 1, - 1, 1)^{T},

(5)

and set

X = (3 u_{1}, u_{2})

and

Y = 21 + 3 u_{1} + u_{2} + \frac{1}{2} u_{3}

. The squared singular values are

9

and

1

. At

λ = 1

, the fitted coordinates along

u_{1}, u_{2}

are

(27/10, 1/2)

, retaining

9/10

and

1/2

of their ordinary least squares values

(3, 1)

. At

λ = 9

they become

(3/2, 1/10)

, retaining

1/2

and

1/10

. The unfittable

u_{3} /2

component contributes

1/4

to every residual sum of squares, giving total values

1/4

59/100

, and

331/100

at penalties

0, 1, 9

. These are constructed fits, not measured prediction results.

Figure 2.
Exact synthetic ridge shrinkage retains the strong singular direction longer than the weak one
. **Details**.

The penalty

λ

is a hyperparameter, so fitting slopes at a fixed value and selecting that value are separate tasks. Use a validation set or K-fold cross-validation to compare candidate penalties by held-out squared-error loss. In each split, compute predictor sample means and scales from that split's training data, apply the same transformation to its validation set, and refit the slopes; using held-out information in preprocessing introduces data leakage. Minimum training residual sum of squares cannot select useful shrinkage: ordinary least squares already attains the smallest possible value.^[ref] After model selection, refit on all training data with the selected penalty. A separate test set or nested cross-validation assesses the entire selection procedure; a low selected validation score alone is no performance guarantee.^[ref]

Saddlepoint conditional likelihood adjustment 2026-10-14

For a regular two-parameter exponential family, condition on the sample mean of the nuisance canonical statistic. Exponential tilting to its observed value and the saddlepoint density approximation give a nuisance-statistic density proportional to

d_{λλ}^{- 1/2}

times its large-deviation exponential. Dividing the joint likelihood by this density gives the profile log-likelihood plus the displayed adjustment, up to a data-only constant. The sign is positive. This construction assumes an interior saddle with positive curvature and a sufficiently regular nonlattice density; conditioning on a lattice statistic requires the corresponding mass approximation.

Saddlepoint density of a Laplace sample mean 2026-10-14

For independent unit-scale Laplace distributions, the cumulant-generating function is

K (t) = - lo g (1 - t^{2})

for

∣ t ∣ < 1

. The saddlepoint density approximation solves

K^{'} (t) = y

, giving

t = y / (1 + 1 + y^{2})

. At this tilt,

K^{''} (t) = D (1 + D)

and

t y = D - 1

, giving the displayed probability density function approximation for the sample mean. Its value at zero is

n / (4 π)

; it need not integrate to one before normalization.

Sample correlation coefficient Created 2026-10-06 Updated 2026-10-14

For paired observations

(x_{i}, y_{i})

, the sample correlation coefficient is

r_{X Y} = \frac{\sum _{i} ( x _{i} - x ) ( y _{i} - y )}{\sum _{i} ( x _{i} - x ) ^{2} \sum _{i} ( y _{i} - y ) ^{2}} .

It is the normalized inner product of the centered data vectors, or the sample covariance divided by the product of the corresponding sample standard deviations. The choice of divisor

n

n - 1

cancels when used consistently in all three quantities. It is undefined if either centered data vector is zero, which occurs when that variable is constant in the observed statistical sample.

For

x = (1, 2, 3, 4)

and

y = (2, 1, 4, 3)

, both sample means are

2.5

. The sum of centered products is

3

, and each sum of centered squares is

5

, hence the sample correlation coefficient is $3/5 = 0.6$ . This is a summary of these four observed pairs; it is not automatically the population Pearson correlation coefficient. Philip Stark, Computing the correlation coefficient explains the centered and standardized calculation.

Sample mean of iid Bernoulli variables 2026-10-14

For

n \geq 1

independent and identically distributed Bernoulli random variables with success probability

p

, the sum

K = \sum_{i} X_{i}

has a binomial distribution. Every length-

n

pattern with

k

successes has probability

p^{k} (1 - p)^{n - k}

, and the binomial coefficient counts its

(k n)

possible success positions. Therefore

P (\overline{X} = k / n) = (k n) p^{k} (1 - p)^{n - k}, E [\overline{X}] = p, Var (\overline{X}) = \frac{p ( 1 - p )}{n} .

The usual endpoint convention includes

p = 0

and

p = 1

as deterministic cases. For

p = 1/2

and

n = 4

, the five probability masses are

1, 4, 6, 4, 1

divided by

16

. The sample mean takes values

0, 1/4, 1/2, 3/4, 1

, with variance

1/16

and standard deviation

1/4

. At

n = 64

, its variance is

1/256

and its standard deviation is

1/16

Figure 1.
Exact sample-mean distributions distinguish fresh IID draws from a copied common coin
. Original exact binomial distribution probabilities for sample sizes $1, 4, 16, 64$ . Blue marks give the mass at each possible mean under independent fair draws. Orange marks give the masses for the same fair draw copied $n$ times: $1/2$ at zero and $1/2$ at one for every $n$ . These are discrete probability masses, not density values or measured histograms. **Details**.

Sample mean sufficiency depends on the model 2026-10-14

For the normal sample sum with known variance, assume independent and identically distributed

X_{i} \sim N (μ, σ^{2})

with fixed known variance

σ^{2} > 0

. The joint probability density factors as

f_{μ} (x) = h (x) (2 π σ^{2})^{- n /2} exp [- \frac{\sum _{i} x _{i}^{2}}{2 σ ^{2}}] g_{μ} (\sum_{i} x_{i}) exp [\frac{μ \sum _{i} x _{i}}{σ ^{2}} - \frac{n μ ^{2}}{2 σ ^{2}}] .

The Fisher-Neyman factorization theorem proves that the sum, and hence the sample mean, is a sufficient statistic for

μ

. Its remaining deviations

X_{i} - \overline{X}

have a joint Gaussian law independent of

μ

and are independent of

\overline{X}

. If both

μ

and

σ^{2}

are unknown, the sample mean alone loses information about variance; the pair

(\sum_{i} X_{i}, \sum_{i} X_{i}^{2})

is sufficient for the full statistical parameter.

A discrete counterexample shows why a sample mean is not automatically sufficient for arbitrary statistical models. Let

X_{1}, X_{2}

be independent and identically distributed on

{0, 1, 2}

with

P_{θ} (X_{i} = 0) = P_{θ} (X_{i} = 2) = θ /2, P_{θ} (X_{i} = 1) = 1 - θ, 0 < θ < 1.

Each observation has mean one. The arrangements

(0, 2), (1, 1), (2, 0)

all give sample mean one, yet

P_{θ} ((X_{1}, X_{2}) = (1, 1) ∣ \overline{X} = 1) = \frac{( 1 - θ ) ^{2}}{( 1 - θ ) ^{2} + θ ^{2} /2} .

This conditional probability is

18/19

θ = 1/4

and

2/11

θ = 3/4

. The conditional distribution therefore still depends on the unknown statistical parameter: this sample mean is not a sufficient statistic. The useful reduction depends on the family of full-data laws, even when its mean stays fixed.

Sample mean under simple random sampling 2026-10-14

Fix

N > 1

real population values

y_{1}, \dots, y_{N}

. Define the finite population mean and two normalizations of the finite population variance by

\overset{ˉ}{Y} = \frac{1}{N} \sum_{i = 1}^{N} y_{i}, S^{2} = \frac{1}{N - 1} \sum_{i = 1}^{N} (y_{i} - \overset{ˉ}{Y})^{2}, σ_{N}^{2} = \frac{1}{N} \sum_{i = 1}^{N} (y_{i} - \overset{ˉ}{Y})^{2} = \frac{N - 1}{N} S^{2} .

The values are fixed; only the selected subset

A

is random. For its sample mean

\overset{y}{ˉ}_{A} = n^{- 1} \sum_{i \in A} y_{i}

, the exact design expectation and variance are

E [\overset{y}{ˉ}_{A}] = \overset{ˉ}{Y}, Var (\overset{y}{ˉ}_{A}) = (1 - \frac{n}{N}) \frac{S ^{2}}{n} = \frac{N - n}{N - 1} \frac{σ _{N}^{2}}{n} .

The sample mean is an unbiased estimator of the finite population mean. At

n = N

, every unit is observed and

\overset{y}{ˉ}_{A} = \overset{ˉ}{Y}

exactly, so the variance is zero. The

N = 1

case is deterministic and is handled directly, since

S^{2}

is undefined then.^[ref]

The finite population correction depends on the baseline. With the

N - 1

normalization it is

1 - n / N

multiplying

S^{2} / n

. With the

N

normalization it is

(N - n) / (N - 1)

multiplying

σ_{N}^{2} / n

, the exact variance for independent uniform sampling with replacement. The standard error uses the square root of the relevant variance factor. Mixing the two normalizations gives a wrong answer even at

n = 1

, when replacement cannot matter. For the six-value example,

\overset{ˉ}{Y} = 7

\sum_{i} (y_{i} - 7)^{2} = 70

S^{2} = 14

and

σ_{N}^{2} = 35/3

, reproducing

14/3

without replacement and

35/6

with replacement.

A short proof exposes the statistical dependence. Let

X_{r}

be the value at position

r

in uniform sequential sampling without replacement, so each

X_{r}

has expected value

\overset{ˉ}{Y}

and variance

σ_{N}^{2}

. Write

d_{i} = y_{i} - \overset{ˉ}{Y}

. Every ordered pair of distinct labels is equally likely, and

\sum_{i} d_{i} = 0

, hence for

r \neq = s

Cov (X_{r}, X_{s}) = \frac{\sum _{i \neq = j} d _{i} d _{j}}{N ( N - 1 )} = \frac{( \sum _{i} d _{i} ) ^{2} - \sum _{i} d _{i}^{2}}{N ( N - 1 )} = - \frac{σ _{N}^{2}}{N - 1} = - \frac{S ^{2}}{N} .

Linearity of expectation proves unbiasedness without requiring independence. The variance of a sum includes all

n (n - 1)

ordered covariance terms, giving

Var (\frac{1}{n} \sum_{r = 1}^{n} X_{r}) = \frac{n σ _{N}^{2} - n ( n - 1 ) σ _{N}^{2} / ( N - 1 )}{n ^{2}} .

(4)

For the original population, each pair of distinct draw positions has covariance

- 7/3

. Drawing a high value leaves a lower mean among the remaining values, balancing subsequent draws. Equal marginal probability distributions therefore coexist with negative covariance.

When

2 \leq n \leq N

, the sample variance

s_{A}^{2} = (n - 1)^{- 1} \sum_{i \in A} (y_{i} - \overset{y}{ˉ}_{A})^{2}

is an unbiased estimator of

S^{2}

. To check this independently, use

\sum_{i \in A} (y_{i} - \overset{y}{ˉ}_{A})^{2} = \sum_{i \in A} (y_{i} - \overset{ˉ}{Y})^{2} - n (\overset{y}{ˉ}_{A} - \overset{ˉ}{Y})^{2}

: its expected value is

n σ_{N}^{2} - n Var (\overset{y}{ˉ}_{A}) = (n - 1) S^{2}

. Thus

(1 - n / N) s_{A}^{2} / n

unbiasedly estimates the variance of the sample mean; its square root estimates the standard error, but need not be unbiased.^[ref]

Figure 1.
Exact mean variance and expected distinct-unit coverage as sample size grows in the six-value population
. **Details**.

The diagram compares variance with the expected fraction of distinct population labels observed. Without replacement this fraction is exactly

n / N

; with independent uniform sampling with replacement, a label is missed with probability

(1 - 1/ N)^{n}

, so linearity of expectation gives expected fraction

1 - (1 - 1/ N)^{n}

. At six draws, replacement covers only about

66.5%

of labels on average and retains sample mean variance

35/18

; sampling all six distinct labels covers the population completely and removes selection uncertainty. This population coverage is distinct from confidence interval coverage probability.

The correction matters when the sampling fraction is appreciable. It calibrates uncertainty about this fixed population under the specified selection mechanism. It does not correct unequal selection weights, missing observations or measurement errors; observing every label also does not eliminate uncertainty about a larger unobserved population. Moreover, an exact variance alone does not guarantee the coverage probability of a proposed confidence interval: a small discrete sampling distribution can make a normal approximation inaccurate.

Sample median Created 2026-09-28 Updated 2026-10-14

The sample median is a central value of an ordered statistical sample. The usual numerical convention selects the middle order statistic for odd sample size and the arithmetic mean of the two central order statistics for even sample size:

X_{n} = {X_{(m + 1)}, (X_{(m)} + X_{(m + 1)}) /2, n = 2 m + 1, n = 2 m .

NIST uses this convention. ^[ref]. For

(0.7, 0.1, 0.9, 0.4, 0.2)

, the sample median is

0.4

; for

(0.1, 0.2, 0.4, 0.7)

, it is

(0.2 + 0.4) /2 = 0.3

, which need not be an observed value.

More generally, every value between the two central order statistics in an even statistical sample is a median of its empirical distribution function. The generalized-inverse sample quantile at

p = 1/2

selects the lower central order statistic in that case, so it can differ from the midpoint convention. Ties are allowed in all these definitions.

The sample median measures the center using sorted positions. Changing the largest observation from

0.9

9

in the five-value example leaves the sample median at

0.4

, while the sample mean rises from

0.46

2.08

. Its sampling distribution still depends on the population probability distribution: the mean

1/2

found for five observations from a uniform distribution is a special calculation, and a sample median's expected value need not equal the population median in general.

For a concrete counterexample, three independent and identically distributed random variables with rate-one exponential distribution have sample median

X_{(2)}

with expected value

1/3 + 1/2 = 5/6

, by independent exponential order-statistic spacings. The population median instead solves

1 - e^{- m} = 1/2

, giving

m = lo g 2 \approx 0.693147

. Thus even the middle order statistic can have mean different from the population median.

Sampling without replacement Created 2026-10-14 Updated 2026-10-14

Sampling without replacement selects distinct labelled units: after a unit is selected it is unavailable for the remaining draws. Distinct labels can carry equal numerical values, so repeated values do not necessarily imply a repeated unit. The rule alone does not specify selection probabilities; a procedure favoring particular units is still sampling without replacement but need not be simple random sampling.

In uniform sequential sampling without replacement, the next label is chosen uniformly among those remaining. For a population of

N

labels, any ordered sequence of

n

distinct labels has probability

1/ [N (N - 1) \dots (N - n + 1)]

. Each size-

n

subset has

n!

possible orders, so its probability is

(n N)^{- 1}

. This gives the fixed-size simple random sampling design. Equivalently, take the first

n

labels of a uniformly random permutation. Restricting a uniformly random permutation to a specified label set preserves uniformity; biased modulo mappings should not replace uniform selection.

The draw positions have identical marginal probability distributions, but exhibit statistical dependence. If the first label is

i

, the conditional probability that the second label is

i

is zero, whereas its marginal probability is

1/ N

. Thus these draws are not independent and identically distributed when

N > 1

and

n > 1

. For a marked subset of labels, counting size-

n

subsets with binomial coefficients gives the hypergeometric distribution. The resulting negative covariance explains the finite population correction for both counts and sample means.

Selection on an extreme first measurement 2026-10-14

Select units with first reading

X > 1

, keep that membership fixed, and measure the same units again. The first reading now has a truncated normal distribution. For a general finite threshold

a

, write

ϕ

for the standard normal density and

Φ

for the standard normal distribution function. Since

ϕ^{'} (x) = - x ϕ (x)

E [X ∣ X > a] = \frac{\int _{a}^{\infty} x ϕ ( x ) d x}{1 - Φ ( a )} = \frac{ϕ ( a )}{1 - Φ ( a )} .

The tower property of conditional expectation lets us average the line

E [Y ∣ X] = 0.6 X

over the selected group.^[ref] At

a = 1

this gives

P (X > 1) E [X ∣ X > 1] E [Y ∣ X > 1] E [Y - X ∣ X > 1] \approx 0.158655, \approx 1.525135, = 0.6 E [X ∣ X > 1] \approx 0.915081, \approx - 0.610054.

The average second reading is below the selection threshold even though every selected first reading exceeded it. This is a group expectation; any finite generated group's sample mean fluctuates around it. Selecting again on

Y > 1

would define a different group and answer a different question.

The latent variable decomposition makes the apparent drop especially clear. Gaussian conditioning gives

E [T ∣ X = x] = 0.6 x

and

E [ε_{1} ∣ X = x] = 0.4 x

. Thus

E [T ∣ X > 1] \approx 0.915081, E [ε_{1} ∣ X > 1] \approx 0.610054, E [ε_{2} ∣ X > 1] = 0.

The first measurement error has zero unconditional mean, but positive selected mean. The second measurement error stays centered because it is independent of selection on

X

. The persistent signal

T

never changed.

Figure 1.
Selection captures an extreme first error while the persistent signal stays fixed
. Original synthetic repeated measurements, not actual measurements. Left: 40 generated units selected by $X > 1$ ; orange trajectories move farther from zero. Right: analytic selected-group expectations decompose the first and second readings into persistent latent variable $T$ and measurement error. Black dots give generated group sample means after selecting from 60,000 synthetic units; their small departures from the bar totals reflect sampling variation. All selections use the first reading only. **Details**.

Shrinking a sample mean with variance proportional to mean squared Created 2026-10-06 Updated 2026-10-14

If independent observations have mean

θ

and variance

θ^{2}

, the scaled sample mean has

MSE (k \overset{ˉ}{X}) = θ^{2} [(k - 1)^{2} + k^{2} / n] .

For

θ \neq = 0

, it improves on the unbiased sample mean precisely when

(n - 1) / (n + 1) < k < 1

. The optimal scale is

k = n / (n + 1)

and its risk is

θ^{2} / (n + 1)

. At

θ = 0

all risks vanish and strict improvement is impossible.

Simple random sampling Created 2026-10-07 Updated 2026-10-14

Here simple random sampling means a random unordered subset

A

of exactly

n

distinct labels from

{1, \dots, N}

, with every size-

n

subset equally likely. The sample size is fixed in advance,

1 \leq n \leq N

, and sampling without replacement prevents duplicate units. This is SRSWOR; some texts also use "simple random sampling" for independent uniform draws with replacement, which is a different design.^[ref]

Every label has inclusion probability

n / N

, but equal inclusion probabilities alone are insufficient. For example, from six labels choose either

{1, 2, 3}

{4, 5, 6}

with probability

1/2

. Each label is included with probability

1/2

, yet eighteen of the twenty size-three subsets are impossible: this is a two-group cluster sampling design. Unequal probability sampling deliberately changes inclusion probabilities, often to improve precision for a particular estimand, and generally requires an appropriately weighted estimator rather than the plain sample mean.

Consider the original mathematical population with labelled values

(2, 4, 6, 8, 10, 12)

, and let

n = 2

. The fifteen equiprobable subsets, followed by their sample means, are

first value 2 46810 {2, 4} : 3 {4, 6} : 5 {6, 8} : 7 {8, 10} : 9 {10, 12} : 11 {2, 6} : 4 {4, 8} : 6 {6, 10} : 8 {8, 12} : 10 {2, 8} : 5 {4, 10} : 7 {6, 12} : 9 {2, 10} : 6 {4, 12} : 8 {2, 12} : 7

Their sample means sum to

105

, giving expected value

105/15 = 7

. Their squared deviations from

7

sum to

70

, giving variance

70/15 = 14/3

. This is an enumeration over hypothetical selections from six stipulated numbers, not measured survey data.

Figure 1.
All fifteen size-two subsets and all thirty-six ordered replacement outcomes from six labelled values
. **Details**.

With independent uniform sampling with replacement, the

6^{2} = 36

ordered outcomes each have probability

1/36

. The two orders

(2, 4)

and

(4, 2)

are distinct outcomes, and

(2, 2)

is now possible. Forgetting order does not make the resulting unordered multisets equiprobable:

{2, 4}

has probability

2/36

but

{2, 2}

has probability

1/36

. The thirty-six sample means still have expected value

7

, but their squared deviations sum to

210

, giving variance

210/36 = 35/6

. Avoiding duplicate labels improves precision for this fixed population.

Singular covariance and nonexistence of a Gaussian maximum likelihood estimate 2026-10-14

For independent

p

-variate multivariate normal observations with unknown mean and unrestricted positive-definite covariance matrix, the centered scatter has rank at most

n - 1

. If it is singular, choose the mean to be the sample mean and shrink the covariance on a null direction of the scatter. The quadratic likelihood term stays fixed while the determinant tends to zero, so the density likelihood tends to infinity. Thus a nonsingular covariance maximum likelihood estimate exists only when the centered scatter is positive definite; for a nonsingular Gaussian population this occurs almost surely exactly when

n > p