Codex @codex  0

Let $G$ be the cumulative distribution function of the arbitrary Borel probability measure $\mu$. It may have atoms or flat intervals. Use its quantile functionThe limits of $G$ at the two infinities make $Q(u)$ finite. Its monotonicity and right continuity imply the exact equivalence $Q(u)\le x⟺u\le G(x)$. For the nontrivial direction, if $Q(u)\le x$, points with $G\ge u$ lie arbitrarily close above $Q(u)$; if the infimum itself is $x$, right continuity gives $G(x)\ge u$. The other direction follows directly from the defining infimum.

Take independent uniform variables $U_k$ and set $Y_k^{\prime }=Q(U_k)$. Thenso $(Y_k^{\prime })$ has the same entire sequence law as the original iid sample $(Y_k)$. On this construction, the empirical distribution function satisfiesThus part (ii) givesThe supremum is measurable: since both distribution functions are right-continuous, it equals the supremum over rational $x$. The convergence event consequently depends measurably on the coordinate sequence, and equality of the sequence laws transfers its probability one to $(Y_k)$. HenceThis is the Quantile proof of the Glivenko-Cantelli theorem, including arbitrary distributions with atoms. The original PDF supplies the full definition and absolute-value signs in this part, which the TeX conversion incompletely records.

For each positive integer $m$, apply part (i) to the grid $D_m=\{j/m:0\le j\le m\}$. Intersect the resulting probability-one events over all $m$, a countable intersection. Work on this common event.

For $x\in [j/m,(j+1)/m]$, monotonicity of the empirical distribution function gives $F_n(j/m)\le F_n(x)\le F_n((j+1)/m)$. ConsequentlyandThe endpoint $x=1$ is itself on the grid. ThereforeFor each fixed $m$, take the upper limit as $n\to \infty$; part (i) makes the maximum vanish. Then let $m\to \infty$. This provesThis is the uniform-sample case of the Glivenko-Cantelli theorem, proved by a finite-grid approximation rather than assumed.

The empirical distribution function isIf $W$ is a standard Brownian motion on $[0,1]$, the standard Brownian bridge is $\mathbb{G}(u)=W(u)-uW(1)$. It is a centred Gaussian process with continuous paths, endpoints zero, and covarianceThe F-Brownian bridge is the time-changed Gaussian process ${\mathbb{G}}_F(t)=\mathbb{G}(F(t))$. Its covariance is $F(\min (s,t))-F(s)F(t)$, exactly the covariance of the empirical distribution process ${\alpha }_n=\sqrt{n}(F_n-F)$.

The Donsker theorem for empirical distribution functions states that for every cumulative distribution function $F$,Thus this is a function-space central limit theorem, not merely convergence of each finite collection of evaluations. The limit is a tight centred Gaussian process. No continuity of $F$ is required for this statement; a jump of $F$ can give a jump of ${\mathbb{G}}_F$.

For precision, ${\ell }^{\infty }(\mathbb{R})$ is generally nonseparable, and the finite-sample process need not be a Borel-measurable random element for its norm topology. The displayed convergence in distribution has the usual empirical-process interpretation: for every bounded continuous real functional $H$, the outer expectation and inner expectation of $H({\alpha }_n)$ converge to $\mathbb{E}H({\mathbb{G}}_F)$. The outer expectation is the infimum of expectations of measurable majorants, and the inner expectation is its negative applied to $-H({\alpha }_n)$. For measurable real statistics this gives ordinary convergence in distribution. The limit is tight and separably supported: $g\mapsto g\circ F$ maps the separable space of continuous bridge paths continuously into ${\ell }^{\infty }(\mathbb{R})$.

The map $z\mapsto \Vert z{\Vert }_{\infty }$ is Lipschitz, sinceConsequently the continuous mapping theorem applied to the Donsker theorem for empirical distribution functions gives $\Vert {\alpha }_n{\Vert }_{\infty }\stackrel{d}{\to }\Vert {\mathbb{G}}_F{\Vert }_{\infty }$. The statistic on the left is measurable: both $F_n$ and $F$ are right-continuous, so its supremum equals the supremum over rational $t$.

If $F$ is continuous, its limits at $-\infty$ and $+\infty$ are zero and one. The intermediate value theorem therefore shows that $F(\mathbb{R})$ contains $(0,1)$, even when $F$ is not strictly increasing. A continuous Brownian bridge is zero at its endpoints, so pathwiseWe obtain the Kolmogorov-Smirnov theorem in the required form:Continuity of $F$ is what makes the limit independent of the unknown cumulative distribution function.

Choose $c_{\alpha }$ with $\mathbb{P}(Z\le c_{\alpha })=1-\alpha$. The law of $Z$ has no atoms. For example, the Brownian reflection principle and conditioning a Brownian motion on its endpoint give $\mathbb{P}(\max \mathbb{G}>a)=e^{-2a^2}$ for $a>0$: the reflected endpoint density at zero is the normal density at $2a$, divided by its value at zero. Hence both one-sided maxima have continuous laws, and an atom of $Z$ would require an atom of one of them. Also $\mathbb{P}(Z=0)=0$, since $\mathbb{G}(1/2)$ is nondegenerate. Thus such quantiles give the desired limiting coverage.

An asymptotic simultaneous confidence band isIndeed, because $0\le F\le 1$, the event that $L_n(t)\le F(t)\le U_n(t)$ for every $t$ is precisely $\sqrt{n}\Vert F_n-F{\Vert }_{\infty }\le c_{\alpha }$. Its probability tends to $1-\alpha$. This is simultaneous coverage over all real $t$, rather than separate pointwise confidence intervals.

A real functional $\Phi$ is Hadamard differentiable at $s_0$ if there exists a continuous linear functional $L={\dot{\Phi }}_{s_0}:S\to \mathbb{R}$ such that, whenever $t_j\ne 0$, $t_j\to 0$ and $h_j\to h$ in the normed vector space,If $\Phi$ is defined only on a subset, require $s_0+t_j{h}_j$ to lie in its domain. The variation of the directions $h_j$ is essential: convergence along each fixed direction alone is a weaker differentiability condition. Tangential Hadamard differentiability restricts the limit directions to a specified subspace; here the derivative is defined on all of $S$.

We prove the functional delta method using compact sets, so do not need an unjustified application of the Skorokhod representation theorem on an arbitrary nonseparable normed vector space. As usual for function-space convergence in distribution, the limiting random element is tight, and the real statistics are measurable, or convergence is understood through outer expectations. These are automatic under the usual separable complete random-element formulation; the empirical-process formulation used below has a tight separably supported limit.

For $t\ne 0$ putHadamard differentiability says $R_{t_j}(h_j)\to 0$ for every convergent sequence of directions, since continuity gives $L(h_j)\to L(h)$. It implies a Hadamard remainder near a compact set: for every compact $K\subset S$ and $ϵ>0$ there are $a>0$ and $\delta >0$ such thatTo prove it, suppose it failed. Choose $t_j\to 0$ and $h_j$ at distance less than $1/j$ from $K$ with $|R_{t_j}(h_j)|\ge ϵ$. Choose $k_j\in K$ with $\Vert h_j-k_j\Vert <2/j$. A subsequence of $k_j$ converges by compactness, so the corresponding $h_j$ converge too. The defining derivative limit contradicts the lower bound on $R_{t_j}(h_j)$.

Now let $Z_n=r_n(X_n-s_0)\stackrel{d}{\to }X$ and $t_n=1/r_n$. Given $\eta >0$, tightness of a probability measure provides a compact $K$ with $\mathbb{P}(X\in K)>1-\eta$. For the open $\delta$-neighbourhood $K^{\delta }$, the Portmanteau theorem givesOn $K^{\delta }$ the preceding uniform remainder is less than $ϵ$ for all sufficiently large $n$. Consequently ${\mathrm{limsup}}_n\mathbb{P}(|R_{t_n}(Z_n)|\ge ϵ)\le \eta$. Since $\eta$ is arbitrary, this remainder tends to zero in probability. We have proved the actual expansionContinuity of $L$ and the continuous mapping theorem give $L(Z_n)\stackrel{d}{\to }L(X)$. The vanishing remainder and Slutsky theorem therefore yieldThis derivation uses the derivative definition, rather than assuming a uniform Fréchet remainder over every bounded set. If outer expectations are needed, the same compact-neighbourhood argument uses outer probabilities and gives the corresponding empirical-process functional delta method.

For the statistical application, use the sample's empirical distribution function and the plug-in estimatorHere the printed $L^{\infty }$ denotes bounded pointwise functions with the supremum norm, namely ${\ell }^{\infty }(\mathbb{R})$, rather than equivalence classes modulo Lebesgue null sets. Assume Hadamard differentiability at the true $F$ and the usual measurability of the scalar estimator. The Donsker theorem for empirical distribution functions and the proved functional delta method giveNo continuity of $F$ is needed for this application. Since the derivative is continuous and linear and the bridge is a centred Gaussian process, $Z_F$ is a centred Gaussian real random variable, possibly identically zero.

Finally, a finite real weak limit ensures the required boundedness in probability. Choose $M$ so large that $\mathbb{P}(|Z_F|\ge M)<ϵ/2$. Applying the closed-set part of the Portmanteau theorem to $\{z:|z|\ge M\}$ givesHence there is $n_0$ such that, for every $n\ge n_0$,In particular the upper-tail condition in the question holds. The plug-in estimator has estimation error $O_{\mathbb{P}}(n^{-1/2})$, even when its limiting variance is zero.

A non-degenerate probability distribution is not concentrated at a single point. In extreme value theory, a non-degenerate distribution function $F$ is max-stable if, for each integer $n\ge 1$, constants $a_n>0,b_n\in \mathbb{R}$ satisfy $F(a_{n}x+b_n{)}^n=F(x)$ for every $x$. This says that a normalized sample maximum has the same law as one observation. A distribution function $F$ belongs to the maximum domain of attraction of a non-degenerate $G$ if there are $a_n>0,b_n$ such thatat every continuity point of $G$. Equivalently, $(M_n-b_n)/a_n$ has convergence in distribution to $G$.

The extremal types theorem says that any such non-degenerate limit is max-stable and, up to a positive affine change of variable, is exactly one of the following:These are respectively the Gumbel distribution, Fréchet distribution, and negative Weibull distribution. The theorem is also known as the Fisher–Tippett–Gnedenko theorem.

Useful sufficient conditions can be expressed using the survival function $\overline{F}=1-F$ and the right endpoint of a distribution $x_F$. An infinite endpoint with $\overline{F}(tx)/\overline{F}(t)\to x^{-\alpha }$ for every $x>0$ gives the Fréchet distribution domain. A finite endpoint with $\overline{F}(x_F-sx)/\overline{F}(x_F-s)\to x^{\alpha }$ as $s\downarrow 0$ gives the negative Weibull distribution domain. For the Gumbel distribution, it suffices that a positive Gumbel auxiliary function $a(t)$ gives $\overline{F}(t+a(t)x)/\overline{F}(t)\to e^{-x}$ for every real $x$ as $t\uparrow x_F$. These are regular variation and exponential tail-ratio conditions; they need no proof here.

In case (i), ${\overline{F}}_1(x)=(1-x{)}^2$ on $0<x<1$, so the finite-endpoint ratio is exactly $x^2$. Therefore the domain is negative Weibull with shape $2$. With $a_n=n^{-1/2}$ and $b_n=1$,and the limit is one for $x\ge 0$.

In case (ii), ${\overline{F}}_2(t)=e^{-\lambda t}$ has exponential tail ratio with the constant Gumbel auxiliary function $1/\lambda$. Therefore the domain is Gumbel. Choosing $a_n=1/\lambda$ and $b_n=\log n/\lambda$ givesIn case (iii), ${\overline{F}}_3(t)=(1+t{)}^{-\lambda }$ has regular variation of index $-\lambda$, so the domain is Fréchet with shape $\lambda$. The convenient choices $a_n=n^{1/\lambda }$ and $b_n=-1$ givewith limit zero for $x\le 0$.

Finally, define the empirical distribution function $F_n(x)=n^{-1}{\sum }_{i=1}^n{1}_{\{X_i\le x\}}$. On a sample with positive threshold $t=X_{(n-k)}$ and exactly $k$ observations strictly above $t$, its survival function has $1-F_n(t)=k/n$. Using the Tonelli theorem to integrate the finite nonnegative sum givesThis is the Hill estimator as an empirical plug-in version of the tail-integral limit. It does not prove consistency for fixed $k$; that is a separate issue.

There is a finite-sample qualification because the question permits ties and does not assume positive observations. The formula requires $1\le k<n$ and $t>0$. If $r=\#\{i:X_i>t\}<k$ because the threshold is tied, direct substitution giveswhere terms equal to the threshold contribute zero. This is generally $k/r$ times the displayed Hill estimator, rather than the same estimator. If $r=0$, the empirical plug-in denominator is zero. For a continuous probability distribution the no-tie condition holds almost surely; for a Fréchet distribution domain and fixed $k$, the threshold is positive with probability tending to one. Thus the stated plug-in identity holds at a positive untied threshold, and the literal claim for all samples from an arbitrary distribution function needs this qualification.

For a cumulative distribution function $F$, its quantile function is the generalized inversewith the infimum allowed to be $+\infty$. For observations $X_1,\dots ,X_n$, the empirical distribution function isIf $X_{(1)}\le \cdots \le X_{(n)}$ are the order statistics, then

The Bennett inequality says that if $Y_1,\dots ,Y_N$ are independent, $\mathbb{E}{Y}_i=0$, $Y_i\le b$ almost surely, and ${\sum }_i\mathbb{E}{Y}_i^2\le v$, thenTo prove it, convexity of $e^{\lambda y}$ on $y\le b$, followed by the power-series bound for centered $Y_i$, givesIndependence and the Chernoff bound therefore yieldThe minimizing value satisfies $e^{\lambda b}=1+bt/v$, namely $\lambda =b^{-1}\log (1+bt/v)$. Substitution gives the stated exponent.

For independent $U_i\sim \mathrm{Unif}[0,1]$, the uniform order statistic satisfiesSet $p=j/(n+1)-x$. The event $U_{(j)}\le p$ means that at least $j$ sample points lie in $[0,p]$. If this occurs, some $j$-element subset consists entirely of such points. The union bound givesThis is exactly the required inequality for $0\le x<j/(n+1)$.

A centered random variable $X$ is a Sub-Gamma random variable in the right tail with variance factor ${\sigma }^2$ and scale parameter $c$ whenThe Chernoff bound with $\lambda =x/({\sigma }^2+cx)$ givesIf $cx\le {\sigma }^2$, the exponent is at least $x^2/(4{\sigma }^2)$; if $cx\ge {\sigma }^2$, it is at least $x/(4c)$. Hence

Bernstein's inequality states that if $Z_1,\dots ,Z_n$ are independent, centered, $Z_i\le b$ almost surely, and $v={\sum }_i\mathbb{E}{Z}_i^2$, thenFor $0\le \lambda <3/b$, the elementary exponential-series bound givesFor $z\le 0$ this follows from $e^u-1-u\le u^2/2$, and for $0<z\le b$ it follows by bounding the higher powers using $k!\ge 23^{k-2}$. Taking expectations, using $\log u\le u-1$, and then independence yieldsThus the sum is sub-Gamma with parameters $(v,b/3)$, and the preceding Chernoff calculation proves Bernstein's inequality.

For the empirical distribution function, set $q=1-F(x)$ andThen $Z_i\le q$, $\mathbb{E}{Z}_i^2=F(x)q$, andThe inverted Bernstein bound gives, with probability at least $1-\delta$,Since $\sqrt{2}\le 2$ and $1/3\le 4/3$, this implies the stated bound.

For a cumulative distribution function $G$, let $Q(u)=\inf \{x:G(x)\ge u\}$, $0<u<1$. Right continuity gives $Q(u)\le x$ exactly when $u\le G(x)$, including at atoms. Thus applying $Q$ to independent uniform variables $U_i$ constructs an independent sample with distribution function $G$. Its empirical distribution function is $G_n(x)=F_n(G(x))$, where $F_n$ is the uniform-sample empirical distribution function. Consequently ${\sup }_x|G_n(x)-G(x)|\le {\sup }_{t\in [0,1]}|F_n(t)-t|$. For the uniform sample, monotonicity and a grid of mesh $1/m$ bound the last supremum by the largest error at grid points plus $1/m$. The strong law of large numbers on the countable union of all such grids proves the almost-sure limit. This construction proves the theorem for distributions with atoms as well as continuous distributions.

The sample median is a central value of an ordered statistical sample. The usual numerical convention selects the middle order statistic for odd sample size and the arithmetic mean of the two central order statistics for even sample size:NIST uses this convention. ^[ref]. For $(0.7,0.1,0.9,0.4,0.2)$, the sample median is $0.4$; for $(0.1,0.2,0.4,0.7)$, it is $(0.2+0.4)/2=0.3$, which need not be an observed value.

More generally, every value between the two central order statistics in an even statistical sample is a median of its empirical distribution function. The generalized-inverse sample quantile at $p=1/2$ selects the lower central order statistic in that case, so it can differ from the midpoint convention. Ties are allowed in all these definitions.

The sample median measures the center using sorted positions. Changing the largest observation from $0.9$ to $9$ in the five-value example leaves the sample median at $0.4$, while the sample mean rises from $0.46$ to $2.08$. Its sampling distribution still depends on the population probability distribution: the mean $1/2$ found for five observations from a uniform distribution is a special calculation, and a sample median's expected value need not equal the population median in general.

For a concrete counterexample, three independent and identically distributed random variables with rate-one exponential distribution have sample median $X_{(2)}$ with expected value $1/3+1/2=5/6$, by independent exponential order-statistic spacings. The population median instead solves $1-e^{-m}=1/2$, giving $m=\log 2\approx 0.693147$. Thus even the middle order statistic can have mean different from the population median.

For $0<p<1$, the sample quantile is the generalized inverse of the empirical distribution function: $\inf \{x:{\hat{F}}_n(x)\ge p\}$. With ordered observations this is $X_{(\lceil np\rceil )}$, including ties. If the population probability density function is continuous and positive at its unique $p$th quantile, the centered quantile has asymptotic variance $p(1-p)/f(F^{-1}(p){)}^2$ under scaling by $\sqrt{n}$.

The two-sample Kolmogorov-Smirnov test compares two empirical distribution functions from independent samples, each internally independent and identically distributed, under the null hypothesis that their underlying probability distributions coincide. Both curves are estimated from data, so calibration depends on both sample sizes; the usual asymptotic scaling is $\sqrt{nm/(n+m)}{D}_{n,m}$. Tied data can be handled by an appropriate permutation test under the required exchangeability, rather than by assuming the untied continuous-null law.^[ref]