= Glivenko-Cantelli theorem
{c}
{wiki}
For <independent and identically distributed> real observations $X_i$ with <cumulative distribution function> $F$, the <empirical distribution function>
$$
F_n(x)=\frac1n\sum_{i=1}^n\mathbf1_{\{X_i\leq x\}}
$$
satisfies the <Glivenko-Cantelli theorem>
$$
\sup_{x\in\mathbb R}|F_n(x)-F(x)|\longrightarrow0\quad\text{almost surely}.
$$
The population law may have atoms; continuity of $F$ is not required. The conclusion is uniform over every threshold, strengthening the separate strong laws for fixed thresholds.<Probability: Theory and Examples (fifth edition) by Rick Durrett>[Section 2.4, Theorem 2.4.9]{ref}
Here is a proof using <inverse transform sampling>. First take <independent and identically distributed>[iid] uniforms and their empirical CDF $G_n$. At the finitely many grid points $j/m$, the <strong law of large numbers> gives $G_n(j/m)\to j/m$ simultaneously. Monotonicity bounds the discrepancy between grid points by the largest grid discrepancy plus $1/m$. Taking countably many $m\to\infty$ proves uniform convergence on $[0,1]$.
For general $F$, construct $X_i=F^{-1}(U_i)$ using the <quantile function>. Then $X_i\leq x$ exactly when $U_i\leq F(x)$, so $F_n(x)=G_n(F(x))$ and the general supremum is bounded by the uniform one. This also handles jumps of $F$. The theorem does not apply merely because an empirical CDF was formed: for repeated copies of the same nondegenerate random observation, the empirical CDF does not approach its population law in this way.
Back to article page
