 Robust estimator

A robust estimator is an estimator whose behavior remains acceptably stable under specified departures from an assumed statistical model. “Stable” must identify a criterion, such as limited bias under contamination, bounded response to a single extreme observation, or resistance to replacing a fraction of the sample. It does not mean that the estimator is accurate under every possible probability distribution.

For example, consider data $(-1,0,1,M)$ with $M\ge 1$. The sample mean is $M/4$, whereas the sample median, using the midpoint convention for an even-sized sample, is $1/2$. Sending $M$ to infinity makes the sample mean diverge without changing that sample median. The example concerns location estimation: the population median and population mean need not coincide outside a symmetric reference model.

One formal model is Huber contamination:where $F$ is a reference probability distribution, $G$ is an unspecified contaminating probability distribution, and $\epsilon$ is its proportion. A robustness claim can bound an estimator’s worst bias over allowed $G$, rather than only calculating its variance when the reference model is exactly true. Peter J. Huber’s original location paper treats this tradeoff systematically.^[ref]

A Huber location estimator illustrates how stability enters the calculation. With a specified positive scale $s$, solveLarge residuals have bounded scores, so one increasingly extreme point cannot contribute an arbitrarily large term to this equation. By contrast, the squared-error score grows linearly with the residual and yields the sample mean. A practical analysis must also state how $s$ is obtained: a scale estimate that itself explodes under outliers can undermine the intended protection.

Different criteria answer different questions. An influence function studies an infinitesimal perturbation of a distribution, while a breakdown point concerns a finite amount of sample corruption. A robust estimator can sacrifice some model-specific statistical efficiency to improve performance away from that model, but the amount depends on the chosen tuning and departure. Reporting the target parameter, reference model and robustness criterion makes the claim testable.