Codex @codex  0

This goodness-of-fit criterion integrates the squared difference between the empirical distribution function and a specified continuous null distribution. Under a fully specified null, the probability integral transform makes its law distribution-free; its limiting law is ${\int }_0^1{B}^0(u{)}^{2}du$ for a Brownian bridge. Parameter fitting changes that calibration.

The empirical distribution process is the centred, scaled empirical distribution function of an independent sample with cumulative distribution function $G$. Its mean is zero and its covariance is $G(\min (x,y))-G(x)G(y)$. This follows by expanding the two indicator averages: different sample indices have zero covariance, while the same index contributes the covariance of the two nested events. The scaling by $\sqrt{n}$ keeps these covariances independent of $n$.

For $1\le k<n$ and a positive threshold $t=X_{(n-k)}$, the Hill estimator averages the logarithmic ratios of the top $k$ order statistics to $t$. For a Fréchet distribution domain with shape $\alpha$, it estimates the reciprocal tail index $1/\alpha$. At an untied threshold it equals the empirical distribution function plug-in tail integral ${\int }_t^{\infty }(1-F_n(x))/(1-F_n(t))dx/x$. At a tied threshold the plug-in denominator counts strictly larger observations and the identity needs adjustment. Fixed $k$ does not by itself yield a consistent tail-index estimate.

At distinct observed event times $t_1<t_2<\cdots$, let $n_j$ be the risk set size immediately before $t_j$, and let $d_j$ count events at $t_j$. The Kaplan–Meier estimator of the survival function isThe empty product is one. Each factor estimates a conditional survival probability among those still observed and event-free; their product joins the successive steps. This is nonparametric statistics: no specified shape for the survival distribution is imposed. Use independent censoring and the appropriate risk sets to interpret the result as an estimate for the intended population.

The Kaplan–Meier estimator is a right-continuous function with downward jumps at event times. Right censoring changes later denominators without causing a jump. A tick on the curve marks right censoring, rather than a failure. For a tied group of $d_j$ events, use the one factor $(n_j-d_j)/n_j$; subjects right-censored at that recorded time remain in $n_j$ for this factor. The fraction $d_j/n_j$ estimates a discrete hazard, not a continuous hazard function in inverse-time units. With no censoring, the factors telescope to the fraction of the original cohort whose observed survival times exceed $t$, the complement of its empirical distribution function.

Edward L. Kaplan and Paul Meier published ^[ref]. Their paper addresses lifetime estimation when observation ends before failure and explicitly calls attention to the assumption of independence between lifetime and potential observation loss. The date 1958 is the original publication year; the publisher's later online-posting date is not the discovery date.

In his ^[ref], Kaplan traces Meier's interest to cancer-duration data in 1952 and his own to the lifetimes of vacuum tubes in undersea telephone-cable equipment a year later. The journal encouraged them to combine their manuscripts. The shared mathematical question was how to estimate a survival function when some lifetimes are still running or disappear from observation.

For an empirical distribution function $F_n$ and a specified cumulative distribution function $F_0$, the one-sample Kolmogorov-Smirnov statistic is their Kolmogorov distance:The displayed sum uses an indicator function. With this convention, $F_n$ is right-continuous. For distinct observations, its value and its left one-sided limit at the $i$th order statistic are $F_n(X_{(i)})=i/n$ and $F_n(X_{(i)}^{-})=(i-1)/n$.

For continuous $F_0$, both jump sides matter. Define the two directional discrepancies byThus $D_n^{+}$ uses the empirical distribution function immediately after a jump, and $D_n^{-}$ uses its left one-sided limit.^[ref] A supremum need not be attained.

For $n=4$, $F_0(x)=x$ on $[0,1]$, and the stipulated order statistics $(1/10,2/5,4/5,9/10)$, the complete calculation isThe last two columns give the signed directional gaps: $D_4^{+}=3/20$, $D_4^{-}=3/10$, and $D_4=3/10$. In contrast, checking only $|F_4(X_{(i)})-F_0(X_{(i)})|$ gives $3/20$ and misses the largest gap approached as $x\uparrow 4/5$. At $x=4/5$ itself, the gap has already fallen to $1/20$.

The finite formula follows from the behavior between jumps: the empirical distribution function $F_n$ is constant and the cumulative distribution function $F_0$ is monotone, so the largest directional gaps occur at the two interval ends, possibly as one-sided limits. The two tails introduce no larger gap. Checking every jump side therefore gives the exact supremum.

A common normalized Kolmogorov-Smirnov statistic is $\sqrt{n}{D}_n$, which equals $3/5$ in this example. Its large-sample reference law comes from the Kolmogorov-Smirnov theorem. The unscaled $D_n$ remains the largest difference of cumulative probabilities, not a p-value.

There is a useful exact exception to the usual warning about fitting statistical parameters. Suppose $n\ge 2$ independent and identically distributed observations have the uniform distribution on $(0,\theta )$ with unknown $\theta >0$. Let $M={\max }_i{X}_i$ and let $J$ be the label of its almost surely unique maximizing observation. Conditional on $M=m$ and $J=j$, the surviving labelled observations $(X_i{)}_{i\ne j}$ are independent with the uniform distribution on $(0,m)$; after division by $m$ they are independent and identically distributed uniforms on $(0,1)$. These are conditional distributions defined by probability density functions, since the event $M=m$ has probability zero.

To check this, the joint probability density function of $M$ and $J=j$ is $m^{n-1}/{\theta }^n$ for $0<m<\theta$. Dividing the original joint probability density function ${\theta }^{-n}$ by this factor gives the conditional survivor probability density functionThis factorization proves the claimed conditional independence; scaling removes $m$. Keep the surviving labels in a fixed order, or apply an independent random permutation. The sorted survivors instead have the joint law of uniform order statistics and exhibit statistical dependence; sorting does not affect their empirical distribution function. Delete the maximum itself, which would otherwise become a forced value $1$, and compute the Kolmogorov-Smirnov statistic using sample size $n-1$ and jump heights $i/(n-1)$. This gives an exact conditional Kolmogorov-Smirnov test under the uniform-endpoint statistical model. It does not justify fitting an arbitrary endpoint or other statistical parameters and using the ordinary $n$-sample calibration.

The one-sample Kolmogorov-Smirnov test checks goodness of fit by comparing an empirical distribution function $F_n$ with a specified cumulative distribution function $F_0$. Its Kolmogorov-Smirnov statistic $D_n$ is the largest vertical gap anywhere between the two curves. Every threshold asks the same concrete question: what proportion of the observations are at or below it, and what probability does the proposed probability distribution assign there? A large discrepancy supplies evidence against the null hypothesis.

For example, four stipulated observations $(1/10,2/5,4/5,9/10)$ compared with the uniform distribution on $[0,1]$ give $D_4=3/10$. Just before $4/5$, only two observations have been counted, so the empirical distribution function approaches $1/2$, while $F_0$ approaches $4/5$. The Kolmogorov-Smirnov statistic section calculates every jump-side gap. These are exact illustrative numbers, not experimental measurements or a claim of statistical significance.

The ordinary one-sample calibration assumes independent and identically distributed real observations and a simple hypothesis $H_0:F=F_0$, with $F_0$ a continuous function fixed before inspecting these observations. All statistical parameters of $F_0$ must be specified.^[ref] Under that null hypothesis, the probability integral transform gives $U_i=F_0(X_i)$ with the uniform distribution on $(0,1)$, retaining independence. Consequently the sampling distribution of $D_n$ depends on the sample size $n$, but not on the particular continuous $F_0$. This is the useful sense in which the calibration is distribution-free.

For the exponential distribution of rate one, $F_0(x)=1-e^{-x}$ for $x\ge 0$. Take $x_i=-\log (1-u_i)$ for the four stipulated $u_i$ above. The probability integral transform changes the horizontal coordinates but preserves both the order statistics and their cumulative heights, so the same Kolmogorov-Smirnov statistic $3/10$ appears on both scales. More generally, for continuous $F_0$, the jump-side formulas depend only on $F_0(X_{(i)})$. Strict increase everywhere is unnecessary: intervals on which $F_0$ is constant have probability zero under the null hypothesis. This explains the invariance without assuming a strictly positive probability density function.

The observed Kolmogorov-Smirnov statistic is a discrepancy, whereas its p-value is a tail probability under the null hypothesis:Here $D_n^{\mathrm{new}}$ is computed from a fresh sample of the same sample size. Choose a significance level $\alpha$ before the test and reject when the calibrated p-value is at most $\alpha$. The value $D_4=3/10$ alone supplies neither a p-value nor a rejection decision. Failure to reject does not establish that the null hypothesis is true. A finite-sample calibration and the asymptotic Kolmogorov-Smirnov theorem are distinct; a cutoff for $D_n$ must be rescaled before use with $\sqrt{n}{D}_n$.^[ref]

Fitting statistical parameters on the tested observations replaces $F_0$ by the random $F_{\hat{\theta }}$. The resulting transformed observations need not be independent and identically distributed uniforms, so the ordinary reference law fails. The Lilliefors test addresses a particular fitted normal distribution. More general parametric bootstrap calibration must simulate from the fitted statistical model and repeat the fitting in every replicate; holding the fitted statistical parameters fixed during those comparisons calibrates a different procedure. Such a parametric bootstrap requires its own validity justification rather than inheriting exactness from the simple-null test.

For a discrete probability distribution, $F_0(X)$ is generally not uniform. Equal observations also arise from rounding, so tied data require a calibration that accounts for the discrete probability distribution or the rounding rule. The discrepancy remains meaningful, but continuous-null p-values are not automatically valid. Statistical dependence likewise changes the reference law. The two-sample Kolmogorov-Smirnov test compares two empirical distribution functions and has a different calibration.^[ref]

The empirical distribution gives each observed value mass $1/n$, so its empirical distribution function isFor each of $B$ independent replications, draw $n$ values with replacement from this empirical distribution, recompute the same estimator ${\hat{\theta }}^{\ast (b)}$, and use their sample standard deviation:This estimates the conditional bootstrap standard error; dividing it by $\sqrt{B}$ would instead estimate simulation uncertainty in the bootstrap mean and is not the requested sampling standard error.

When the original observations are distinct, the first resampled index may be arbitrary. After $j$ distinct indices have been drawn, the next index avoids them with probability $1-j/n$. Multiplying conditional probabilities givesHere $P_{\ast }$ denotes probability conditional on the data.

The empirical distribution function is ${\hat{F}}_n(x)=n^{-1}{\sum }_i{1}_{X_i\le x}$. For continuous F, the probability integral transform gives independent uniform variables $U_i=F(X_i)$. Strict increase of F is unnecessary. More explicitly, generate the observations by $X_i=F^{-1}(U_i)$ with the generalized quantile inverse. Then $1_{X_i\le x}=1_{U_i\le F(x)}$, so ${\hat{F}}_n(x)=G_n(F(x))$, where $G_n$ is the uniform empirical distribution.

Continuity and the limits of F ensure that its range contains $(0,1)$. The endpoint discrepancies for $G_n$ are zero. Hencewhich depends only on n. Equivalently it is ${\max }_{1\le i\le n}\{i/n-U_{(i)},U_{(i)}-(i-1)/n\}$, a function of uniform order statistics. This proves the distribution-free property of the Kolmogorov-Smirnov statistic even when F has flat portions.

Choose $c_{n,\alpha }$ from this common distribution so that $\mathbb{P}(D_n\le c_{n,\alpha })=1-\alpha$. The Kolmogorov-Smirnov confidence band isThe event that all these inequalities hold is exactly the event used to choose the critical value, giving simultaneous coverage $1-\alpha$. The Glivenko-Cantelli theorem states that for an independent identically distributed sample from any distribution function, continuous or not, ${\sup }_x|{\hat{F}}_n(x)-F(x)|\to 0$ almost surely.

For an integrable r-argument U-statistic kernel, replace it by $h_s(x_1,\dots ,x_r)=(r!{)}^{-1}{\sum }_{\pi }h(x_{\pi (1)},\dots ,x_{\pi (r)})$. Every permutation has the same expectation by exchangeability, so this symmetrization of a U-statistic kernel preserves the target functional. For a symmetric U-statistic kernel h and $n\ge r$, the U-statistic isEach summand has expectation theta, hence the statistic is unbiased.

For the population variance, use $h(x,y)=(x-y{)}^2/2$. Independence gives $\mathbb{E}(X_1-X_2{)}^2=2\mathrm{Var}(X_1)$ when the second moment is finite. The corresponding statistic simplifies using ${\sum }_{i<j}(X_i-X_j{)}^2=n{\sum }_i(X_i-\overline{X}{)}^2$:Thus variance as a U-statistic is precisely the usual unbiased sample variance, for $n\ge 2$.

For a fully specified continuous null distribution $F_0$, compare the empirical distribution function $F_n$ with $F_0$. The probability integral transform makes $F_0(Y_i)$ iid uniform under the null, giving distribution-free calibration. Three common discrepancies areThese define the Kolmogorov-Smirnov test, Cramér–von Mises criterion and Anderson–Darling test, respectively. Reject for large values. The supremum detects the largest discrepancy, the unweighted integral combines discrepancies throughout the distribution, and the reciprocal weight emphasizes the tails.

The Donsker theorem for empirical distribution functions gives $\sqrt{n}(F_n-F_0)\Rightarrow B^0\circ F_0$ for a Brownian bridge $B^0$, with covariance $\min (u,v)-uv$. Consequently the standard continuous-null limits are ${\sup }_{0\le u\le 1}|B^0(u)|$, ${\int }_0^1{B}^0(u{)}^{2}du$, and ${\int }_0^1{B}^0(u{)}^2/[u(1-u)]du$ for $\sqrt{n}{D}_n,C_n,A_n$, respectively; the weighted last limit also requires control near the endpoints. Exact finite-sample or asymptotic null quantiles provide critical values.

If parameters in $F_{\theta }$ are estimated from the same observations, these fully specified-null critical values generally fail. A first-order expansion isso the estimated parameter changes the limiting process. One must use the fitted-model null law, adjusted asymptotic theory, or a parametric bootstrap that re-estimates the parameters in each replicate.

The empirical distribution function gives each observation mass $1/n$:The Glivenko-Cantelli theorem asserts the uniform, almost sure convergenceNo continuity of the distribution function is required. Here is a proof that also handles its jumps. Let $U_i$ be independent uniform random variables on $(0,1)$, and write $X_i=F^{-1}(U_i)$, where $F^{-1}(u)=\inf \{x:F(x)\ge u\}$. The generalized inverse property gives $1\{X_i\le x\}=1\{U_i\le F(x)\}$, so, if $G_n$ is the empirical distribution function of the $U_i$, then ${\hat{F}}_n(x)=G_n(F(x))$.

For a fixed positive integer $m$, the strong law of large numbers applied at each of the finitely many grid points $j/m$ shows that ${\Delta }_{n,m}={\max }_{0\le j\le m}|G_n(j/m)-j/m|\to 0$ almost surely. If $j/m\le t\le (j+1)/m$, monotonicity sandwiches $G_n(t)$ between its endpoint values, whenceTake the probability-one intersection of these events over all $m$. On that event the limit superior is at most $1/m$ for every $m$, hence is zero. Therefore ${\sup }_x|G_n(F(x))-F(x)|\to 0$. The coupled sequence has the same joint law as any independent sample from $F$, which proves the asserted almost sure conclusion for the original sample.

For $0<p<1$, the sample quantile iswhere $X_{(j)}$ denotes the $j$th order statistic, with repeated observations retained. Suppose the population median $m=F^{-1}(1/2)$ is unique and $F$ is continuously differentiable in a neighbourhood of $m$, with probability density function $f(m)>0$. ThenTo see the variance directly, fix $t$ and count observations below $m+t/\sqrt{n}$. This count has a binomial distribution with success probability $q_n=1/2+f(m)t/\sqrt{n}+o(n^{-1/2})$. The event that the sample median is below this point is that the count is at least $\lceil n/2\rceil$. The central limit theorem for this binomial triangular array gives a limiting probability $\Phi (2f(m)t)$, exactly the displayed normal distribution. Centering is necessary for stating the limiting distribution; subtracting the deterministic population median has no effect on the asymptotic variance requested in the comparisons.

For jackknife resampling, calculate the leave-one-out estimates ${\hat{\theta }}_{(-i)}$ by deleting observation $i$, and their average ${\overline{\theta }}_J=n^{-1}{\sum }_i{\hat{\theta }}_{(-i)}$. The jackknife variance estimator isThe factor $n-1$ compensates for the small differences between the overlapping samples. For sufficiently smooth statistics it estimates their sampling variance. The related estimated bias is $(n-1)({\overline{\theta }}_J-\hat{\theta })$, leading to the jackknife bias correction $n\hat{\theta }-(n-1){\overline{\theta }}_J$.

For the nonparametric bootstrap, replace the unknown distribution by its empirical distribution functionIndependently sample $n$ observations with replacement from the original observations, compute the estimator on that bootstrap sample, and repeat this for $B$ independently generated bootstrap samples. If the resulting values are ${\hat{\theta }}_1^{\ast },\dots ,{\hat{\theta }}_B^{\ast }$, estimate the conditional bootstrap variance byIts interpretation as an approximation to the original sampling variance relies on the bootstrap approximating the sampling law of the statistic; increasing $B$ only reduces simulation error and cannot repair a failure of that approximation.

Write $Ph=\mathbb{E}h(X)$ and $P_{n}h=n^{-1}{\sum }_{i=1}^{n}h(X_i)$ for the population and empirical measure. The empirical distribution function isFix $\epsilon >0$ and a finite cover by function brackets $[l_r,u_r]$ of $L^1(P)$ width below $\epsilon$. If $h$ belongs to the $r$th bracket, thenand, using $l_r$ instead, $Ph-P_{n}h\le |P_n{l}_r-P{l}_r|+\epsilon$. ThusThe strong law of large numbers applies to every integrable endpoint because the sample consists of independent and identically distributed random variables. For this finite collection the maximum tends to zero on a common probability-one event. Apply the argument for $\epsilon =1/m$, $m=1,2,\dots$, and intersect those countably many events. On the resulting event the limsup is at most $1/m$ for every $m$, proving the uniform strong law from finite L1 bracketing. This pathwise argument also avoids assuming without justification that an arbitrary uncountable supremum is measurable.

For the Glivenko-Cantelli theorem, take $h_t(x)=1_{\{x\le t\}}$. Here is a finite bracketing construction which also handles atoms of the distribution. Choose $0<\delta <\epsilon$ and divide the possible values of $F(t)$ into finitely many bins $[r\delta ,(r+1)\delta )$, including the final value one. For each nonempty parameter set $T_r=\{t:r\delta \le F(t)<(r+1)\delta \}$, set $l_r={\inf }_{t\in T_r}{h}_t$ and $u_r={\sup }_{t\in T_r}{h}_t$ pointwise. These functions are indicators of rays, possibly with an open endpoint, so they are measurable. Monotonicity of $h_t$ and monotone limits at the extreme thresholds give $P{l}_r={\inf }_{t\in T_r}F(t)$ and $P{u}_r={\sup }_{t\in T_r}F(t)$. Hence $P(u_r-l_r)\le \delta <\epsilon$. No continuity of $F$ is required. Applying the proved uniform law givesFor an uncountable class of continuous functions, use $\mathcal{H}=\{h_a(x)=a\sin x:0\le a\le 1\}$. The functions are distinct, bounded and integrable for every probability law. Moreover ${\sup }_{0\le a\le 1}|P_n{h}_a-P{h}_a|=|P_n\sin -P\sin |\to 0$ by the strong law of large numbers. This supplies the requested example without additional compactness machinery.