 Robust statistics

Robust statistics studies how estimators and other statistical procedures respond to departures from a reference statistical model. Such departures can include a small fraction of outliers, a heavier-tailed probability distribution, or an incorrect assumed shape. A robustness statement specifies both the permitted departure and the quantity whose change is controlled.

For data $(0,1,2,3,M)$ with $M\ge 4$, the sample median remains $2$, whereas the sample mean is $(6+M)/5$ and grows without bound. This does not make the sample median an estimator of every desired expected value: the mean and median of an asymmetric probability distribution can differ. The estimand must therefore be specified before judging robustness.

Influence functions describe infinitesimal changes, breakdown points describe large finite-sample failures, and minimax asymptotic bias compares worst cases over a neighborhood. M-estimators connect these questions with an explicit loss function. Peter J. Huber's 1964 location paper developed a systematic treatment of estimation under contamination.^[ref]