Breakdown point Created 2026-09-24 Updated 2026-11-05
A breakdown point measures how much contamination can make an estimator fail without bound. It specifies a contamination mechanism and a notion of failure. For scalar location under replacement contamination, failure means that replacing some observations can move the estimated location arbitrarily far.
The sample mean of observations can be made arbitrarily large by changing just one observation. For an odd sample of size , the sample median stays between original data values after at most replacements, but replacements can send it to infinity.
Two finite-sample conventions record either the first fraction allowing unbounded error or the largest fraction still guaranteed bounded. For the sample mean they are and ; for that odd-sample sample median they are and . The replacement breakdown point must state which convention is used. Both describe a finite change, whereas an influence function describes an infinitesimal perturbation.
Huber location estimator Created 2026-09-24 Updated 2026-10-18
More generally, fix a known scale and a dimensionless threshold before fitting. For observations , a Huber location estimator is a minimizer of the summed Huber loss, equivalently a root of its estimating equation:Here is the Huber loss and the Huber score. The physical clipping distance is , in the observations' units; the initial formula uses . A large observation keeps contributing a bounded signed score rather than being discarded. The original clipped location criterion appears in Peter J. Huber's 1964 paper.[ref]
Robust Estimation of a Location Parameter by Peter J. Huber. §4(iii), printed p. 79: quadratic-central, linear-tail criterion and clipping around the fitted location
For the original data , take and . At , the standardized differences and their Huber scores areTheir sum is zero, so the Huber location estimator is . Four scores remain in the central linear region, making the root unique. By comparison, the sample mean is , and the sample median, using the midpoint of the two middle observations, is . Agreement with the sample median is a feature of these data, rather than an identity between the two estimators.
The geometry explains both existence and possible nonuniqueness. The Huber loss is a convex function, so is a convex function of and its derivative is . For , tends to infinity as grows, hence attains a minimum; its zero derivative characterizes every minimizer. Between clipping transitions, the summed score has slope , where is the number of observations strictly within of . A root with is unique. If , a flat interval is possible: for data with , every is a minimizer. A complete single-valued estimator then needs a selection rule, such as the interval's midpoint.
A bounded Huber score limits one observation's contribution to the estimating equation. The influence function of a Huber location estimator additionally divides by the population's central mass, so the size of its bound depends on the reference probability distribution. Clipping does not by itself make the bias of an estimator zero or prove a universal breakdown point; the separate local influence versus replacement breakdown argument specifies what fixed scale permits. Estimating from the same data changes the procedure and requires separate analysis.
Fix a scale and cutoff . Define the population Huber location estimator by the rootHere is the Huber score. It has Lipschitz continuity, so dominated convergence justifies differentiation when puts no mass at . Then , and the nonzero derivative identifies the local population root. The implicit function theorem applies away from the contaminating score's kinks; its Lipschitz continuity also gives the first-order expansion at those kinks. Differentiating its contaminated equation yieldsThe normalization matters: the Huber score alone is not the influence function. The gross-error sensitivity is , while the rejection point of an influence function is infinite because the clipped tails stay nonzero. The original clipped estimating criterion and its derivative normalization also appear in the population analysis of M-estimators.[ref]
Robust Estimation of a Location Parameter by Peter J. Huber. Peter J. Huber, Robust Estimation of a Location Parameter, §3, p. 78: derivative normalization; §4(iii), p. 79: clipped location criterion and central-mass denominator
Scale sets what counts as a large residual. For a normal distribution with fixed , symmetry gives and , where is the standard normal distribution function. If the scale is instead a statistical functional , differentiation of the joint equation adds the scale contribution:Under a symmetric baseline and an odd Huber score, , so scale estimation drops out at first order. This cancellation needs regularity of and does not protect finite contamination changes: an inflated standard deviation enlarges the physical cutoff . A robust choice, such as a suitable Scale M-estimator, needs its own influence function and breakdown point checked. Fixing a numeric cutoff in raw units also sacrifices scale equivariance when units change.[ref]
Robust Estimation of a Location Parameter by Peter J. Huber. Peter J. Huber, Robust Estimation of a Location Parameter, §11, pp. 95-96: estimating location when normal-component scale and contamination fraction are unknown
Past exam of the mathematics course of the University of Cambridge 2002 iii Paper 38 4 Solution Created 2026-10-03 Updated 2026-10-14
A statistical functional is a map defined on a class of probability distributions. Its associated functional statistic is , where is the empirical distribution and each is a Dirac measure. For example, yields the sample mean. The influence function records the derivative in the direction of point contamination:It is a local, infinitesimal robustness diagnostic, not a description of arbitrary large contamination.
Suppose is differentiable in a sense strong enough to linearize it at the empirical distribution, and that its derivative in a signed direction is . Averaging the directions over gives the zero direction, so linearity implies . The resulting asymptotic linear representation isIf , the central limit theorem and the negligible remainder giveThus the asymptotic variance is . For a vector functional the integral is . Convergence of the actual finite-sample variance to this leading expression additionally requires control of second moments of the remainder, or an appropriate uniform integrability condition; convergence in distribution alone does not provide that. Existence of a contamination derivative by itself also does not guarantee the required empirical linearization.
Several complementary robustness measures follow. Gross-error sensitivity is . Under contamination , the first-order bias is , whose worst absolute coefficient is ; bounded influence therefore bounds infinitesimal gross-error bias. The local-shift sensitivity is , equal to when the influence function is smoothly differentiable. It measures the effect of moving a contaminant a small distance. The rejection point of an influence function is , using centered, standardized coordinates; it is infinite if there is no such cutoff. A finite cutoff completely rejects sufficiently extreme observations to first order. These measures address different aspects of robustness. The integral of squared influence controls asymptotic precision and efficiency, while breakdown point concerns finite amounts of contamination and is not determined by bounded influence alone.
For the quantile, assume , a unique solution , and a density continuous and positive at . Define quantiles of the contaminated distribution by generalized inversion, since a distribution with a point mass need not attain as an equality. If , its indicator is constant near for sufficiently small contamination, and differentiatingat zero gives . HenceAt the exceptional point , the generalized-inverse quantile remains exactly : below the contaminated cumulative distribution function is below , and its value at exceeds . Its actual derivative there is zero. The usual step-function expression is an almost-everywhere version, which is all that the asymptotic variance needs. Its mean is zero, and its second moment isTherefore the influence function of a quantile gives asymptotic variance and gross-error sensitivity . Its jump makes local-shift sensitivity infinite, and its nonzero constant tails make its rejection point infinite. If or the quantile is not unique, this regular derivation does not apply.
Past exam of the mathematics course of the University of Cambridge 2003 iii Paper 41 4 Solution Created 2026-10-03 Updated 2026-10-14
A functional statistic has the form , where is the empirical distribution and is a statistical functional on a class of distributions. It separates the population quantity from its sample estimate. Fisher consistency for a model means for every model parameter; it does not alone establish consistency of the empirical estimate without continuity and probabilistic conditions.
The influence function measures infinitesimal contamination at a point:when this derivative exists. A bounded influence function limits first-order sensitivity to an arbitrarily remote observation.
An M-estimator is a minimizer of an empirical loss, or a chosen consistent solution of an estimating equation . Its population functional solves . Put and , assumed finite and nonzero. Differentiating the contaminated population equation givessince the unperturbed expected score is zero. HenceUnder differentiability, consistency and the uniform local laws needed for expansion of the empirical equation,The central limit theorem now yields asymptotic normality with sandwich variance of an M-estimatorThus the leading variance of itself is the displayed variance divided by . For a vector parameter, replace by the derivative matrix and use .
In a location model with known scale , choose . For a symmetric reference distribution, an odd score is Fisher consistent. With and positive , the formulas becomeThe Gaussian likelihood score gives the efficient sample mean under normality, but its unbounded influence is vulnerable to large outliers. The Huber score is linear centrally and bounded in the tails, balancing normal-model efficiency against contamination sensitivity. Its denominator is for a continuous law. The sign score gives the median and needs a nonsmooth derivation using the density at the median, rather than incorrectly setting its ordinary derivative to zero. Redescending scores can reject very extreme observations, but may have multiple roots and require a consistent root-selection rule.
Thus appropriate choices seek Fisher consistency, a nonzero stable derivative, finite variance, reasonable reference-model efficiency and bounded influence. Robust scale estimation may be needed for the tuning constant; joint location-scale estimation uses the matrix influence calculation rather than treating an estimated scale as known. Bounded influence describes local robustness and is not by itself a global breakdown point theorem.
Robust estimator 2026-11-05
A robust estimator is an estimator whose behavior remains acceptably stable under specified departures from an assumed statistical model. “Stable” must identify a criterion, such as limited bias under contamination, bounded response to a single extreme observation, or resistance to replacing a fraction of the sample. It does not mean that the estimator is accurate under every possible probability distribution.
For example, consider data with . The sample mean is , whereas the sample median, using the midpoint convention for an even-sized sample, is . Sending to infinity makes the sample mean diverge without changing that sample median. The example concerns location estimation: the population median and population mean need not coincide outside a symmetric reference model.
One formal model is Huber contamination:where is a reference probability distribution, is an unspecified contaminating probability distribution, and is its proportion. A robustness claim can bound an estimator’s worst bias over allowed , rather than only calculating its variance when the reference model is exactly true. Peter J. Huber’s original location paper treats this tradeoff systematically.[ref]
Robust Estimation of a Location Parameter by Peter J. Huber. Introduction, contaminated normal location models; section 4(iii), clipped location criterion
A Huber location estimator illustrates how stability enters the calculation. With a specified positive scale , solveLarge residuals have bounded scores, so one increasingly extreme point cannot contribute an arbitrarily large term to this equation. By contrast, the squared-error score grows linearly with the residual and yields the sample mean. A practical analysis must also state how is obtained: a scale estimate that itself explodes under outliers can undermine the intended protection.
Different criteria answer different questions. An influence function studies an infinitesimal perturbation of a distribution, while a breakdown point concerns a finite amount of sample corruption. A robust estimator can sacrifice some model-specific statistical efficiency to improve performance away from that model, but the amount depends on the chosen tuning and departure. Reporting the target parameter, reference model and robustness criterion makes the claim testable.
Robust statistics Created 2026-09-24 Updated 2026-11-05
Robust statistics studies how estimators and other statistical procedures respond to departures from a reference statistical model. Such departures can include a small fraction of outliers, a heavier-tailed probability distribution, or an incorrect assumed shape. A robustness statement specifies both the permitted departure and the quantity whose change is controlled.
For data with , the sample median remains , whereas the sample mean is and grows without bound. This does not make the sample median an estimator of every desired expected value: the mean and median of an asymmetric probability distribution can differ. The estimand must therefore be specified before judging robustness.
Influence functions describe infinitesimal changes, breakdown points describe large finite-sample failures, and minimax asymptotic bias compares worst cases over a neighborhood. M-estimators connect these questions with an explicit loss function. Peter J. Huber's 1964 location paper developed a systematic treatment of estimation under contamination.[ref]
Robust Estimation of a Location Parameter by Peter J. Huber. Introduction and location estimation under contaminated normal models
Scale M-estimator Created 2026-09-28 Updated 2026-11-05
A Scale M-estimator estimates the spread of centered data by solvingfor a chosen function . If the center is estimated, residuals replace . The function and its normalization specify the population scale being estimated; an arbitrary choice need not have a positive or unique root.
For known center zero and at least one nonzero observation, choosing gives , the root mean square. This example is highly sensitive to an outlier. Bounded choices of can instead limit an observation's contribution to the estimating equation, but their influence function and breakdown point still require analysis.
If is an even function and the positive root is unique, multiplying the data by multiplies the root by : substitute into the equation. This is scale equivariance, the expected response to changing measurement units.