For positive integers , the th order statistic from independent uniform distributions on has Beta distribution . Selecting the observations below and above a value gives its probability density function proportional to , with normalizing factor .
Box plot 2026-10-14
A box plot compresses a statistical sample into a box spanning its lower and upper quartiles , a line at its median , and two whiskers. It makes location and central spread easy to compare across groups on a common value axis. The box represents the central half in terms of quantile levels; with ties or interpolated quartiles, it need not contain exactly half the observations. Whisker endpoints require a stated convention: they can be the observed extremes, or they can stop short of separately marked observations. [ref]
Here is one explicitly toy dataset, reused under histogram: twelve invented, unitless values, with repeated values counted separately, already arranged as order statistics:
For this box plot we choose the interpolated quartiles defined under quartile, using R's type-7 rule. This gives
The interquartile range is . Our whisker convention uses the fences
The whiskers end at the smallest and largest observations in the closed interval : here and . Values strictly outside the fences receive separate outlier markers: here only . An observation exactly on a fence is retained within the whisker under this convention. The fences are thresholds, not observed whisker endpoints; the upper whisker is at , not . R's documented whisker rule likewise includes observations no more than box lengths away, although its box uses hinges, which can differ from our type-7 quartiles. [ref]
Figure 1.
Toy box plot: quartiles define the box, observations define the whiskers
. The twelve invented values are shown individually below the box plot, with repeated values stacked. Type-7 quartiles are ; the interquartile range is . Dashed fences at and determine whiskers at and and a separate marker at . The fence lines and raw-value strip are explanatory additions, not required parts of every box plot. Details.
A separate outlier marker is a prompt to investigate, not a finding that the value is bad data and not an instruction to delete it. A distant observation can be a recording error, a legitimate unusual outcome, or evidence that the assumed statistical model misses some behavior. Check its provenance and context before deciding how to analyze it. No measurement-error claim is possible for our invented : we deliberately put it there to demonstrate the rule. [ref]
The beauty of a box plot is its compact comparison of ordered positions rather than every observation. That compression also loses information: many different statistical samples have the same quartiles and whiskers, so the box alone cannot reveal every cluster or gap that a histogram might show. For example, replacing the two invented values by and leaves the type-7 quartiles, whiskers and separate marker unchanged. Displaying the individual values alongside the box plot is especially helpful for such a small statistical sample. Always state both the quartile rule and the whisker rule; changing either can change which values are marked.
Data refinement 2026-10-31
Implementation of an abstract state and operations using a more concrete representation, justified by a coupling invariant. For example, an insertion-history list may be implemented by a sorted array when clients observe only occurrence counts and order statistics. The observable behavior, rather than insertion order internally, must agree.
Let be descending order statistics, with and nonnegative decreasing prizes. Decompose the prize vector into awards of to each of the best players. The corresponding truthful multi-unit auction charges each winner the next value . Its total payment is . Revenue equivalence transfers the expected payment to the all-pay contest, because the interim allocations and lowest-type utilities coincide. Summing the layers proves the formula.
Failure probabilities for two independent lifetimes Created 2026-10-18 Updated 2026-10-20
For independent survival times with cumulative distribution functions , the probability both components have failed by time is
The probability at least one has failed is
The first equality uses independence of random variables; the second applies it to the complementary event that both survive. These are the distributions of the two extreme order statistics.
Marginal distributions alone do not determine these probabilities. Without independence, the Fréchet inequalities give , and inclusion-exclusion gives .
Hill estimator Created 2026-10-06 Updated 2026-10-14
For and a positive threshold , the Hill estimator averages the logarithmic ratios of the top order statistics to . For a Fréchet distribution domain with shape , it estimates the reciprocal tail index . At an untied threshold it equals the empirical distribution function plug-in tail integral . At a tied threshold the plug-in denominator counts strictly larger observations and the identity needs adjustment. Fixed does not by itself yield a consistent tail-index estimate.
For independent and identically distributed rate- exponential random variables, let and be their order statistics. The gaps are independent and have rates . After the first clock rings, the memoryless property makes the surviving clocks fresh independent exponentials, independently of the elapsed minimum and the winning label. Induction gives the result. Equivalently, the ordered joint probability density is on ; changing to gaps gives a product of exponential densities because .
Interquartile range 2026-10-14
The interquartile range measures the width between the lower and upper quartiles, in the same units as the observations. It describes central spread rather than the distance between the observed extremes. In our toy box plot, ; moving the largest value from to would leave the quartiles and interquartile range unchanged under the chosen rule, because the order statistics used in their interpolation would not change.
For an empirical distribution function and a specified cumulative distribution function , the one-sample Kolmogorov-Smirnov statistic is their Kolmogorov distance:
The displayed sum uses an indicator function. With this convention, is right-continuous. For distinct observations, its value and its left one-sided limit at the th order statistic are and .
For continuous , both jump sides matter. Define the two directional discrepancies by
Thus uses the empirical distribution function immediately after a jump, and uses its left one-sided limit.[ref] A supremum need not be attained.
For , on , and the stipulated order statistics , the complete calculation is
The last two columns give the signed directional gaps: , , and . In contrast, checking only gives and misses the largest gap approached as . At itself, the gap has already fallen to .
Figure 1.
Both jump sides determine the exact Kolmogorov-Smirnov statistic
. The empirical distribution function for four stipulated rational observations is compared with the uniform distribution on . Open circles show left one-sided limits; filled circles show values at jumps. The green gap is and the magenta gap is the limiting . Details.
The finite formula follows from the behavior between jumps: the empirical distribution function is constant and the cumulative distribution function is monotone, so the largest directional gaps occur at the two interval ends, possibly as one-sided limits. The two tails introduce no larger gap. Checking every jump side therefore gives the exact supremum.
A common normalized Kolmogorov-Smirnov statistic is , which equals in this example. Its large-sample reference law comes from the Kolmogorov-Smirnov theorem. The unscaled remains the largest difference of cumulative probabilities, not a p-value.
There is a useful exact exception to the usual warning about fitting statistical parameters. Suppose independent and identically distributed observations have the uniform distribution on with unknown . Let and let be the label of its almost surely unique maximizing observation. Conditional on and , the surviving labelled observations are independent with the uniform distribution on ; after division by they are independent and identically distributed uniforms on . These are conditional distributions defined by probability density functions, since the event has probability zero.
To check this, the joint probability density function of and is for . Dividing the original joint probability density function by this factor gives the conditional survivor probability density function
This factorization proves the claimed conditional independence; scaling removes . Keep the surviving labels in a fixed order, or apply an independent random permutation. The sorted survivors instead have the joint law of uniform order statistics and exhibit statistical dependence; sorting does not affect their empirical distribution function. Delete the maximum itself, which would otherwise become a forced value , and compute the Kolmogorov-Smirnov statistic using sample size and jump heights . This gives an exact conditional Kolmogorov-Smirnov test under the uniform-endpoint statistical model. It does not justify fitting an arbitrary endpoint or other statistical parameters and using the ordinary -sample calibration.
Kolmogorov-Smirnov test 2026-10-14
The one-sample Kolmogorov-Smirnov test checks goodness of fit by comparing an empirical distribution function with a specified cumulative distribution function . Its Kolmogorov-Smirnov statistic is the largest vertical gap anywhere between the two curves. Every threshold asks the same concrete question: what proportion of the observations are at or below it, and what probability does the proposed probability distribution assign there? A large discrepancy supplies evidence against the null hypothesis.
For example, four stipulated observations compared with the uniform distribution on give . Just before , only two observations have been counted, so the empirical distribution function approaches , while approaches . The Kolmogorov-Smirnov statistic section calculates every jump-side gap. These are exact illustrative numbers, not experimental measurements or a claim of statistical significance.
The ordinary one-sample calibration assumes independent and identically distributed real observations and a simple hypothesis , with a continuous function fixed before inspecting these observations. All statistical parameters of must be specified.[ref] Under that null hypothesis, the probability integral transform gives with the uniform distribution on , retaining independence. Consequently the sampling distribution of depends on the sample size , but not on the particular continuous . This is the useful sense in which the calibration is distribution-free.
For the exponential distribution of rate one, for . Take for the four stipulated above. The probability integral transform changes the horizontal coordinates but preserves both the order statistics and their cumulative heights, so the same Kolmogorov-Smirnov statistic appears on both scales. More generally, for continuous , the jump-side formulas depend only on . Strict increase everywhere is unnecessary: intervals on which is constant have probability zero under the null hypothesis. This explains the invariance without assuming a strictly positive probability density function.
Figure 1.
The probability integral transform preserves the Kolmogorov-Smirnov gap on exponential and uniform scales
. The same four stipulated points are shown as on the left and on the right. Open circles indicate left limits of the empirical distribution function, filled circles its values. The gap is approached from below the third observation on both scales. Details.
The observed Kolmogorov-Smirnov statistic is a discrepancy, whereas its p-value is a tail probability under the null hypothesis:
Here is computed from a fresh sample of the same sample size. Choose a significance level before the test and reject when the calibrated p-value is at most . The value alone supplies neither a p-value nor a rejection decision. Failure to reject does not establish that the null hypothesis is true. A finite-sample calibration and the asymptotic Kolmogorov-Smirnov theorem are distinct; a cutoff for must be rescaled before use with .[ref]
Fitting statistical parameters on the tested observations replaces by the random . The resulting transformed observations need not be independent and identically distributed uniforms, so the ordinary reference law fails. The Lilliefors test addresses a particular fitted normal distribution. More general parametric bootstrap calibration must simulate from the fitted statistical model and repeat the fitting in every replicate; holding the fitted statistical parameters fixed during those comparisons calibrates a different procedure. Such a parametric bootstrap requires its own validity justification rather than inheriting exactness from the simple-null test.
For a discrete probability distribution, is generally not uniform. Equal observations also arise from rounding, so tied data require a calibration that accounts for the discrete probability distribution or the rounding rule. The discrepancy remains meaningful, but continuous-null p-values are not automatically valid. Statistical dependence likewise changes the reference law. The two-sample Kolmogorov-Smirnov test compares two empirical distribution functions and has a different calibration.[ref]
Laplace location and scale maximum likelihood Created 2026-10-18 Updated 2026-10-25
Suppose , , are independent and identically distributed with Laplace distribution , where . For realized values , the likelihood function and log-likelihood function are
For fixed , maximum likelihood estimation of the location therefore minimizes the sum of absolute losses. This connects the Laplace distribution to robust statistics: each observation contributes a cost growing linearly with its distance, whereas squared-error loss grows quadratically.
Why does a sample median minimize ? Between successive order statistics, moving a little to the right increases each distance to a value on the left and decreases each distance to a value on the right. Hence the slope is the number on the left minus the number on the right. It changes from negative to positive at the middle. More precisely, if , all minimizing locations are
For an even sample size, the objective is flat throughout the interval between the two central order statistics. The midpoint convention for the sample median selects one solution; it does not prove uniqueness. [ref] This is the finite-sample counterpart of median minimizes expected absolute loss.
If , maximizing over gives
The derivative is positive below this value and negative above it. Moreover the maximized likelihood function is proportional to wherever , so the same minimizing locations give the joint maximum-likelihood estimates. [ref] The scale estimate divides by ; this formula is not an assertion that it is an unbiased estimator.
The boundary case matters: if all observations equal , choose . Then and as . There is no finite joint maximum-likelihood estimate in the parameter space . In particular, this always occurs for . Writing does not produce an admissible Laplace distribution.
Let be an integer and let be independent and identically distributed random variables with an exponential distribution of rate . Their sample maximum is nonnegative. Substituting the exponential distribution's cumulative distribution function into the distribution functions of independent minima and maxima gives the exact law
For example, at each observation is below the threshold with probability , but all observations are below it with probability . Thus an individual observation's median becomes an increasingly unlikely upper bound for the whole batch. The general order statistic density in Siegrist's treatment, with , also gives this law on integration. [ref].
Solving gives the exact quantile function, for ,
In a toy model with , the median maximum is seconds for , seconds for , and seconds for . All three use the exact distribution function; the increase comes from taking a maximum over more observations. The case connects to the existing maximum of two exponentials as a scaled sum.
Figure 1.
Exact cumulative distribution functions of exponential sample maxima, with rate one per second and median markers
. Details.
The Gumbel distribution explains the logarithmic shift visible here: after subtracting and measuring deviations in units of , the distribution functions approach one fixed shape. The next derivation makes that convergence in distribution explicit.
For observations , the nearest neighbour distance of order at is the th order statistic of the distances . An atomless distance distribution function ensures ties occur with probability zero.
Normal Q-Q plot 2026-10-14
A normal Q-Q plot compares order statistics with standard normal quantiles . For a normal distribution with , the population quantile function is , so the diagnostic sample relation is
The appropriate reference is a line with an intercept and positive slope, not necessarily the identity line. A line can be fitted through two central sample quantiles; R's qqline uses the quantile levels and by default. Neither that line nor its fitting rule is part of the definition of a normal Q-Q plot. [ref]
To isolate curvature, retain the same 19 plotting positions and construct two more ordered sequences:
The are continuous uniform distribution quantiles on , while the apply the hyperbolic sine to the reference quantiles. Both examples are symmetric about zero. At ranks , is approximately and is approximately . Again, these are constructed values rather than random draws or real data.
Figure 1.
Normal Q-Q curvature distinguishes compressed and stretched outer quantiles
. Left: the constructed uniform distribution quantiles bend inward relative to the central tangent . Right: the constructed values bend outward relative to the central tangent . Both panels use 19 standard normal quantiles at . Dashed lines are analytic central tangents, not fitted lines or uncertainty bands; only the chosen finite range is illustrated. Details.
Read these shapes relative to their central slope. The uniform distribution example has , with derivative ; at the upper endpoint the tangent predicts , while . Its lower end lies above the tangent and its upper end below it: the outer quantiles are compressed. The hyperbolic sine example has ; its tangent predicts at the upper endpoint, while . Its lower end lies below the tangent and its upper end above it: the outer quantiles are stretched. Unequal departures at the two ends can suggest skewness. The direction of these descriptions assumes that reference quantiles are horizontal and observed values vertical. The graphs illustrate finite-range shape differences; a sample pattern cannot establish an asymptotic tail probability law or membership in a heavy-tailed distribution class.
A nearly straight normal Q-Q plot is a diagnostic, not a proof of a normal distribution or a formal hypothesis test. Random statistical samples fluctuate, particularly at extreme order statistics, and apparent curvature depends on sample size, plotting positions and the reference line. A calibrated hypothesis test or uncertainty band requires its own assumptions and must account for parameters estimated from the same observations. Sorting also discards the original observation order: rearranging a time series leaves its Q-Q plot unchanged, so the plot cannot establish independence.
For a normal linear model, R's diagnostic normal Q-Q plot uses standardized regression residuals, also called internally studentized residuals. [ref] Even with independent errors, the regression residuals have covariance matrix , where is the hat matrix; fitting parameters generally creates dependencies. Dividing by a shared estimated standard deviation creates further dependence, and the distribution of an internally studentized Gaussian residual is not exactly the standard normal distribution. The normal Q-Q plot therefore helps assess error shape alongside other diagnostics rather than certifying independent normal random variables.
Order statistic Created 2026-09-24 Updated 2026-10-14
An order statistic is a value selected by its position in a sorted statistical sample. For real observations , write
The th order statistic is the th smallest value, counting repeated values separately. The parentheses distinguish sorted position from the original observation label: and need not be equal. This definition permits ties and requires neither independence nor a continuous probability distribution. For a random statistical sample, each order statistic is itself a random variable and a statistic.
For example, the five fixed observations
sort to
Their original indices in sorted order are , so . Their ranks of observations, listed in original observation order, are . Here is the minimum, is the sample maximum, and is the sample median. These are deliberately chosen observations, rather than simulated draws.
Figure 1.
Five observations: original indices, sorted positions, and the third order statistic
. Details.
Order statistics put extremes and typical values in one language: the first and last locate the minimum and sample maximum, while intermediate positions give sample quantiles and the sample median. Under the existing generalized-inverse convention, the sample quantile at level is . For the five observations, the sample quantiles at are . The distinction between a fixed observed value and its sampling distribution lets us quantify how much an extreme or sample median would vary across repeated statistical samples.
The useful counting identity is
For independent and identically distributed random variables with a continuous cumulative distribution function , the count on the right has binomial distribution with parameters , giving
This cumulative distribution function is a binomial distribution tail: a question about sorting becomes a question about counting successes in Bernoulli trials. The counting proof also works for probability distributions with atoms and ties, although a probability density function need not then exist. For a uniform order statistic, on , which makes the formula particularly explicit.
Use the first order statistic and correct its bias:
This is a measurable function of the sample. Under , write . It lies in , and independence gives
For a bounded nonnegative random variable, integrating the identities and proves the tail integral formula for moments by Tonelli's theorem. Hence
Thus is an unbiased estimator. Its quadratic risk is the variance of , independent of :
Taking the supremum over all real and then the infimum over estimators gives
The final inequality follows from . Together with part (i), this proves that the minimax risk has order for all . In the last line of the source, the request for the upper-bound constant calls it ; here it is denoted consistently with the upper-bound inequality.
For , the probability integral transform makes iid uniform on . Continuity of is enough; strict increase is unnecessary. Their extreme order statistics have joint density
choose the observations giving the two extremes and put the remaining between them. Transform to and integrate from zero to . The uniformized sample range therefore has
Equivalently . For , the range is identically zero instead.
The exact failure probability is
Thus the exact integer sample size is the least making this quantity at most . For with , it tends to . This function decreases strictly from one to zero on positive , because its derivative is . For there is therefore exactly one positive solution of
This gives the extreme-order-statistic tolerance interval sample-size rule
When is small, , so a useful first refinement is . This is an approximation, not an exact integer guarantee; use the preceding exact failure probability to check rounding and very stringent tail requirements. The interval here encloses population content, rather than estimating a fixed distribution parameter.
The empirical distribution function is . For continuous F, the probability integral transform gives independent uniform variables . Strict increase of F is unnecessary. More explicitly, generate the observations by with the generalized quantile inverse. Then , so , where is the uniform empirical distribution.
Continuity and the limits of F ensure that its range contains . The endpoint discrepancies for are zero. Hence
which depends only on n. Equivalently it is , a function of uniform order statistics. This proves the distribution-free property of the Kolmogorov-Smirnov statistic even when F has flat portions.
Choose from this common distribution so that . The Kolmogorov-Smirnov confidence band is
The event that all these inequalities hold is exactly the event used to choose the critical value, giving simultaneous coverage . The Glivenko-Cantelli theorem states that for an independent identically distributed sample from any distribution function, continuous or not, almost surely.
For an integrable r-argument U-statistic kernel, replace it by . Every permutation has the same expectation by exchangeability, so this symmetrization of a U-statistic kernel preserves the target functional. For a symmetric U-statistic kernel h and , the U-statistic is
Each summand has expectation theta, hence the statistic is unbiased.
For the population variance, use . Independence gives when the second moment is finite. The corresponding statistic simplifies using :
Thus variance as a U-statistic is precisely the usual unbiased sample variance, for .
The first right endpoint is the smallest of independent pair maxima. A maximum exceeds with probability , so for ,
Taking the limit gives the requested, correctly normalized result
Thus converges to a continuous nonnegative limit with survival function . In particular the scaled first right endpoint is tight. This calculation uses maxima of pairs, not the first order statistic of all individual endpoints, which would have a different scale. The converted TeX loses the formula in this part; the displayed limit has been recovered directly from the PDF.
Write . The answer is
The sum of exponential variables with linearly increasing rates has the distribution of the maximum of independent rate- exponential random variables. To prove this identity, start independent rate- exponential clocks. The first rings after a rate- exponential time; the memoryless property leaves fresh independent clocks. Repeating this argument shows that the successive order statistic gaps are independent, with rates . Their sum has the distribution of . Therefore
For , , and hence
If , subtracting the two tails gives the interval exponent , since the tail at is exponentially smaller. Also , so the Markov inequality gives in probability. Local lower bounds at positive points and at zero follow; the tail estimate gives the closed-set upper bound exactly as in part (a). This proves the full large deviation principle.
There is also a deduction from part (b). For fixed and , a rate- exponential random variable is stochastically smaller than a rate- one. Coupling by the same uniform variables therefore gives with independent additional copies. For fixed and sufficiently large , the upper-tail exponent from part (b) is . Let to obtain ; the lower tail estimate matches it. The exact maximum identity additionally supplies all local lower bounds.