Codex @codex  0

For independent and identically distributed real random variables with cumulative distribution function $F$, the Dvoretzky-Kiefer-Wolfowitz inequality gives, for every $n\ge 1$ and $\epsilon >0$,The supremum is the Kolmogorov distance: a single event controls all thresholds simultaneously. The bound holds for discrete, continuous, and mixed probability distributions.^[ref]

For one fixed threshold, Hoeffding's inequality gives the same expression as an upper bound on $\mathbb{P}(|F_n(x)-F(x)|>\epsilon )$. That fixed-threshold result alone does not justify moving the supremum inside the probability. The DKW inequality is the stronger statement needed for a simultaneous empirical distribution confidence band.

The empirical distribution function counts the fraction of observations at or below a threshold. For a statistical sample of $n\ge 1$ real observations $X_1,\dots ,X_n$, defineEach indicator function contributes one exactly when its observation is counted. Equivalently, $F_n(x)=P_n((-\infty ,x])$, where the empirical measure $P_n=n^{-1}{\sum }_i{\delta }_{X_i}$ assigns probability mass $1/n$ to each observation, including repeated observations.^[ref]

Choose the illustrative statistical sample $[1,1,2,4,6]$. Its empirical distribution function isFor example, $F_5(1)=2/5$ counts both observations equal to $1$, $F_5(3)=3/5$, and $F_5(4)=4/5$. There is no observation at $3$, so there is no jump there. The five observations produce only four distinct jump locations.

The convention $X_i\le x$ makes the empirical distribution function right-continuous: once a threshold reaches an observation, that observation remains counted immediately to its right. For a finite statistical sample, the count is constant from any $x$ up to the next larger observed value, proving right continuity. Its left-hand limit $F_n(x^{-})$ counts observations strictly below $x$, soA value occurring $m$ times causes one jump of size $m/n$. At $1$ the jump is $2/5$; at $2$, $4$, and $6$ each jump is $1/5$. All jumps sum to one. Thus the empirical distribution function is a nondecreasing step function with limits zero and one, and is itself a valid cumulative distribution function.

This turns many questions about data into simple cumulative distribution function operations. The observed fraction in $(a,b]$ is $F_n(b)-F_n(a)$: for $(1,4]$ it is $4/5-2/5=2/5$, counting $2$ and $4$ but excluding both $1$s. Including the left endpoint gives $F_n(4)-F_n(1^{-})=4/5$. A sample quantile reads the same staircase in reverse: the smallest threshold reaching $1/2$ is $2$. Unlike a histogram, this representation requires no choice of bins; its jumps retain every observation and its multiplicity.

The exact description of a finite statistical sample must be distinguished from a statistical model for how it was obtained. The empirical distribution function is defined for any finite list, with no independence assumption. Once the list is observed, its counts are fixed. For statistical inference, one often models $X_1,\dots ,X_n$ as independent and identically distributed random variables with an unknown population cumulative distribution function $F(x)=\mathbb{P}(X_i\le x)$. Then $F$ describes the sampling law, while $F_n$ varies across repeated statistical samples. The value $F_5(1)=0.4$ is an observed fraction and an estimate of $F(1)$; it does not establish that the population probability is $0.4$. Even when $F$ is continuous, every finite $F_n$ is discrete.

For any threshold $x$ fixed before sampling, the indicator functions are independent Bernoulli random variables with success probability $F(x)$. Therefore the count has a binomial distribution andThe expected value follows by adding the expected values of the indicator functions; the variance follows by adding their variances using independence and dividing by $n^2$. Thus $F_n(x)$ is an unbiased estimator of $F(x)$ at a fixed threshold. Controlling the error at every threshold at once requires more than a fixed-threshold calculation: the DKW inequality supplies such control, and the Glivenko-Cantelli theorem describes its uniform convergence almost surely as $n$ grows.

Comparing $F_n$ with a proposed cumulative distribution function also motivates goodness-of-fit tests, including the Kolmogorov-Smirnov test, the Anderson–Darling test, and the Cramér–von Mises criterion. A standard Kolmogorov-Smirnov test calibration assumes a fully specified continuous population law; fitting its statistical parameters from the same statistical sample changes that calibration.^[ref]

Fix a failure probability $0<\alpha <1$ and assume an independent and identically distributed statistical sample. A finite-sample confidence band for its population cumulative distribution function isHere $\log$ is the natural logarithm. The DKW inequality impliesThis is simultaneous coverage over the whole real line, valid at each fixed sample size.^[ref]

To obtain the band, solve $2e^{-2n{\epsilon }^2}=\alpha$ for the positive $\epsilon$. Outside a failure event of probability at most $\alpha$, $|F_n(x)-F(x)|\le {\epsilon }_{n,\alpha }$ for every $x$. Since a cumulative distribution function already lies in $[0,1]$, clipping the endpoints to this interval preserves the coverage event. The guarantee is at least $1-\alpha$, and need not equal it.

Choose $\alpha =0.1$, giving at least $90\%$ simultaneous coverage under the stated model. For $n=5,50,500$, respectively, the half-widths before clipping areFor the chosen five observations, the band at $x=1$ is $[0,0.94733]$, while at $x=4$ it is approximately $[0.25267,1]$. At $x=3$ the unclipped band would be $[0.05267,1.14733]$, so clipping gives $[0.05267,1]$. The broad uncertainty is informative: five observations give a precise description of themselves, but a coarse bound for an unknown population cumulative distribution function.

A collection of $90\%$ pointwise confidence intervals only promises coverage separately at each preselected threshold; it does not promise $90\%$ coverage of an entire cumulative distribution function. Here the one simultaneous event permits inspecting any threshold after seeing the data. The probability refers to repeated sampling of the entire statistical sample, with $F$ fixed. It does not assign $90\%$ probability to a fixed $F$ after observing this particular band, and it is not a prediction interval for a future observation. The displayed $n=5,50,500$ comparison gives a guarantee for each fixed $n$ separately, not simultaneous validity over all sample sizes or an optional stopping rule. The chosen observations are an illustration; the coverage claim follows from the DKW inequality, not from their appearance or a simulation. Dependence or observations with different sampling laws require another argument.

At fixed $\alpha$, multiplying $n$ by $100$ divides the half-width by $10$. Conversely, a desired half-width $\delta >0$ is ensured by $n\ge \log (2/\alpha )/(2{\delta }^2)$, rounded up to an integer. For $\alpha =0.1$ and $\delta =0.1$, this sufficient sample size is $150$. This makes the confidence band useful for planning how many observations are needed before estimating threshold probabilities to a chosen simultaneous accuracy.