= Kolmogorov-Smirnov statistic
{c}
{title2=$D_n=\sup_x|F_n(x)-F(x)|$}
For an <empirical distribution function> $F_n$ and a specified <cumulative distribution function> $F_0$, the one-sample <Kolmogorov-Smirnov statistic> is their <Kolmogorov distance>:
$$
F_n(x)=\frac1n\sum_{j=1}^n\mathbf1_{\{X_j\leq x\}},\qquad
D_n=\sup_{x\in\mathbb R}|F_n(x)-F_0(x)|.
$$
The displayed sum uses an <indicator function>. With this convention, $F_n$ is <right-continuous>. For distinct observations, its value and its left <one-sided limit> at the $i$th <order statistic> are $F_n(X_{(i)})=i/n$ and $F_n(X_{(i)}^-)=(i-1)/n$.
For <continuous> $F_0$, both jump sides matter. Define the two directional discrepancies by
$$
D_n^+=\sup_x(F_n(x)-F_0(x))
=\max_{1\leq i\leq n}\left\{\frac{i}{n}-F_0(X_{(i)})\right\},
$$
$$
D_n^-=\sup_x(F_0(x)-F_n(x))
=\max_{1\leq i\leq n}\left\{F_0(X_{(i)})-\frac{i-1}{n}\right\},\qquad
D_n=\max(D_n^+,D_n^-).
$$
Thus $D_n^+$ uses the <empirical distribution function> immediately after a jump, and $D_n^-$ uses its left <one-sided limit>.<NIST and SEMATECH e-Handbook of Statistical Methods/1.3.5.16. Kolmogorov-Smirnov Goodness-of-Fit Test>{ref} A <supremum> need not be attained.
For $n=4$, $F_0(x)=x$ on $[0,1]$, and the stipulated <order statistics> $(1/10,2/5,4/5,9/10)$, the complete calculation is
$$
\begin{array}{c|c|c|c|c|c}
i&X_{(i)}&F_4(X_{(i)}^-)&F_4(X_{(i)})&i/4-F_0(X_{(i)})&F_0(X_{(i)})-(i-1)/4\\\hline
1&1/10&0&1/4&3/20&1/10\\
2&2/5&1/4&1/2&1/10&3/20\\
3&4/5&1/2&3/4&-1/20&3/10\\
4&9/10&3/4&1&1/10&3/20
\end{array}
$$
The last two columns give the signed directional gaps: $D_4^+=3/20$, $D_4^-=3/10$, and $D_4=3/10$. In contrast, checking only $|F_4(X_{(i)})-F_0(X_{(i)})|$ gives $3/20$ and misses the largest gap approached as $x\uparrow4/5$. At $x=4/5$ itself, the gap has already fallen to $1/20$.
\Image[/kolmogorov-smirnov-ecdf-jump-gaps.png]
{title=Both jump sides determine the exact Kolmogorov-Smirnov statistic}
{description=The <empirical distribution function> for four stipulated rational observations is compared with the <uniform distribution> on $[0,1]$. Open circles show left <one-sided limits>; filled circles show values at jumps. The green gap is $D_4^+=3/20$ and the magenta gap is the limiting $D_4^-=3/10$.}
The finite formula follows from the behavior between jumps: the <empirical distribution function> $F_n$ is constant and the <cumulative distribution function> $F_0$ is <monotone>, so the largest directional gaps occur at the two interval ends, possibly as <one-sided limits>. The two tails introduce no larger gap. Checking every jump side therefore gives the exact <supremum>.
A common normalized <Kolmogorov-Smirnov statistic> is $\sqrt nD_n$, which equals $3/5$ in this example. Its large-sample reference law comes from the <Kolmogorov-Smirnov theorem>. The unscaled $D_n$ remains the largest difference of cumulative <probabilities>, not a <p-value>.
There is a useful exact exception to the usual warning about fitting <statistical parameters>. Suppose $n\geq2$ <independent and identically distributed> observations have the <uniform distribution> on $(0,\theta)$ with unknown $\theta>0$. Let $M=\max_iX_i$ and let $J$ be the label of its almost surely unique maximizing observation. Conditional on $M=m$ and $J=j$, the surviving labelled observations $(X_i)_{i\ne j}$ are <independent> with the <uniform distribution> on $(0,m)$; after division by $m$ they are <independent and identically distributed> uniforms on $(0,1)$. These are <conditional distributions> defined by <probability density functions>, since the event $M=m$ has <probability> zero.
To check this, the joint <probability density function> of $M$ and $J=j$ is $m^{n-1}/\theta^n$ for $0<m<\theta$. Dividing the original joint <probability density function> $\theta^{-n}$ by this factor gives the conditional survivor <probability density function>
$$
m^{-(n-1)}\prod_{i\ne j}\mathbf1_{\{0<x_i<m\}}.
$$
This factorization proves the claimed <conditional independence>; scaling removes $m$. Keep the surviving labels in a fixed order, or apply an <independent> random <permutation>. The sorted survivors instead have the joint law of <uniform order statistics> and exhibit <statistical dependence>; sorting does not affect their <empirical distribution function>. Delete the maximum itself, which would otherwise become a forced value $1$, and compute the <Kolmogorov-Smirnov statistic> using <sample size> $n-1$ and jump heights $i/(n-1)$. This gives an exact conditional <Kolmogorov-Smirnov test> under the uniform-endpoint <statistical model>. It does not justify fitting an arbitrary endpoint or other <statistical parameters> and using the ordinary $n$-sample calibration.
Back to article page
