Influence function ($\mathrm{IF}(x;T,F)$)

An influence function asks how much a population answer moves when an infinitesimal fraction of its probability distribution is relocated to a specified point. It distinguishes the unbounded response of the mean from the bounded response of the median and the Huber location estimator. The normal-location influence curves and finite contamination bias of normal location below make that difference quantitative.

Let $T$ be a real-valued statistical functional, $F$ a reference probability measure, and ${\delta }_x$ the Dirac measure concentrated at $x$. Along the epsilon-contamination neighborhood path $F_{\epsilon ,x}=(1-\epsilon )F+\epsilon {\delta }_x$, define the right derivativeThe name influence curve refers to the same object. The derivative is in the signed direction ${\delta }_x-F$, so the calculation removes baseline mass as well as adding a Dirac measure. The definition is a limit for each fixed $x$; it does not automatically give a remainder uniform over arbitrarily distant $x$.^[ref]

The statistical functional must be defined on these contaminated probability measures, and the right derivative must exist and be finite. For a mean, a finite first absolute moment suffices. For a median, require a unique baseline median and a probability density function continuous and strictly positive there. For a smooth M-estimator, require an isolated population root, justified differentiation of its expected value, and a nonzero derivative with respect to the fitted parameter. These are specific conditions for the formulas below. Existence of the pointwise influence function alone proves neither an asymptotic linear representation nor pathwise differentiability of a statistical functional; each needs control of its own remainder and admissible statistical paths.

For the statistical functional $T(F)=\mu ={\mathbb{E}}_{F}X$, assuming ${\mathbb{E}}_F|X|<\infty$, the contaminated mean is exactlyThus the influence function of the mean is $x-\mu$, and its gross-error sensitivity is infinite on an unbounded sample space. The linear approximation is exact here: any fixed positive contamination fraction allows an arbitrarily large change by moving $x$ far enough. No finite second moment is needed for this derivative, although a finite variance is needed for the usual central limit theorem for the sample mean.

For comparison, the influence function of the sample median iswhen the median $m$ is unique and the probability density function $f$ is positive and continuous at $m$. The sign function records which side of $m$ receives the contaminating mass. The influence function of a quantile supplies the short derivation. With the generalized-inverse quantile convention, contamination exactly at $x=m$ leaves the median at $m$, giving derivative zero there. If $f(m)$ is very small, this bounded influence function can still be large; if $f(m)=0$, the displayed formula is unavailable.

Fix a scale $s>0$ and cutoff $k>0$. Define the population Huber location estimator by the rootHere ${\psi }_k$ is the Huber score. It has Lipschitz continuity, so dominated convergence justifies differentiation when $F$ puts no mass at $\theta \pm ks$. Then $A={\mathbb{P}}_F(|X-\theta |<ks)$, and the nonzero derivative identifies the local population root. The implicit function theorem applies away from the contaminating score's kinks; its Lipschitz continuity also gives the first-order expansion at those kinks. Differentiating its contaminated equation yieldsThe normalization matters: the Huber score alone is not the influence function. The gross-error sensitivity is $sk/A$, while the rejection point of an influence function is infinite because the clipped tails stay nonzero. The original clipped estimating criterion and its derivative normalization also appear in the population analysis of M-estimators.^[ref]

Scale sets what counts as a large residual. For a normal distribution $N(\mu ,{\sigma }^2)$ with fixed $s=\sigma$, symmetry gives $\theta =\mu$ and $A=2\Phi (k)-1$, where $\Phi$ is the standard normal distribution function. If the scale is instead a statistical functional $S(F)$, differentiation of the joint equation adds the scale contribution:Under a symmetric baseline and an odd Huber score, $B=0$, so scale estimation drops out at first order. This cancellation needs regularity of $S$ and does not protect finite contamination changes: an inflated standard deviation enlarges the physical cutoff $ks$. A robust choice, such as a suitable Scale M-estimator, needs its own influence function and breakdown point checked. Fixing a numeric cutoff in raw units also sacrifices scale equivariance when units change.^[ref]

For $F=N(\mu ,{\sigma }^2)$, put $z=(x-\mu )/\sigma$. The influence function of the mean, influence function of the sample median, and influence function of a Huber location estimator with fixed $s=\sigma$ are respectivelyThe first grows without bound, the second jumps at the median, and the third clips continuously. For $k=1.345$, $2\Phi (k)-1\simeq 0.821375$ and the normalized Huber tail is about $1.63750\sigma$. This illustrates the optimal bounded influence function for normal location without asserting that this cutoff is optimal for every contamination model.

Both panels are theoretical normal distributions, with no sampled or observed data. Doubling $\sigma$ doubles the vertical influence function scale and the horizontal clipping distance. Changing $\mu$ translates the curves. The isolated dot marks the actual median derivative at the center; connecting its two tails by a vertical segment would misrepresent that value. Bounded influence does not mean every small residual has less effect than on the mean: the central Huber slope is $1/(2\Phi (k)-1)>1$, a price paid for reducing sensitivity in the tails.

Consider the population statistical model $F_{\epsilon ,x}=(1-\epsilon )N(0,1)+\epsilon {\delta }_x$. The following are deterministic statistical functional calculations, not observed contamination experiments. Compare the actual location bias with the linear approximation $\epsilon \mathrm{IF}(x;T,F)$. The mean bias is exactly $\epsilon x$. For positive $x$ exceeding the shifted median and $0\le \epsilon <1/2$, the generalized-inverse median bias is exactlyThis formula applies to both $x=1$ and $x=8$ throughout the plotted interval $0\le \epsilon \le 0.1$. If the contaminating atom lies inside the quantile jump containing probability $1/2$, the generalized-inverse median instead equals that atom.

For the Huber location estimator with $s=1$ and $k=1.345$, solve its actual population equation, rather than substituting the influence function:Writing $a=t-k$ and $b=t+k$, integration of the normal distribution giveswhere $\varphi$ is the probability density function of $N(0,1)$. Its derivative is $-[\Phi (t+k)-\Phi (t-k)]<0$, so the mixture score is strictly decreasing for $\epsilon <1$. For $\epsilon <1/2$, its limits at negative and positive infinity have opposite signs even with arbitrary distant point contamination. The displayed examples use the unique root, evaluated by the bisection method to numerical precision.

At $x=8$ and $\epsilon =0.01$, the exact mean, median, and Huber location estimator biases are $0.080000$, $0.012660$, and $0.016541$; the influence function predictions are $0.080000$, $0.012533$, and $0.016375$. At $\epsilon =0.05$, they become $0.400000$, $0.066012$, and $0.086241$, versus predictions $0.400000$, $0.062666$, and $0.081875$. The bounded alternatives control the effect of the distant atom, but their actual response curves bend away from the tangent.

The nearby atom $x=1$ gives a useful comparison: at $\epsilon =0.05$, the Huber location estimator bias is $0.060237$, slightly below its linear approximation $0.060874$, whereas the median bias is still $0.066012$. The direction and size of approximation error depend on the contaminating location. Even accurate local influence calculations must specify $F$, the scale, the cutoff, and the actual contamination fraction.

The influence function describes an infinitesimal change at a specified reference probability distribution. The replacement breakdown point asks how many observations in a particular finite sample can be replaced before an estimator becomes unbounded. The finite-sample maximum bias records the intervening changes. These are different questions: a bounded influence function at one baseline is not a general bound over all epsilon-contamination neighborhoods, nor a proof of a positive replacement breakdown point.^[ref]

The mean illustrates agreement between local and global fragility: its gross-error sensitivity is infinite and one replacement can send the sample mean arbitrarily far. The median has finite gross-error sensitivity under the stated positive-density assumptions, while its sample median withstands fewer than about half the observations being replaced under ordinary tie conventions. Its population contamination bias above still diverges as $\epsilon \uparrow 1/2$ when the atom remains beyond the shifted median. Here $x$ must also move outward; taking that limit with a fixed finite atom would invalidate the displayed quantile formula.

For scalar location with a fixed positive scale, the Huber score has a direct global argument: if strictly more than half of the sample stays fixed, then beyond every retained observation by more than $ks$, their saturated scores outweigh all possible replaced scores. Every root therefore stays in a bounded interval. This is a separate finite-replacement argument; estimating scale changes its premises. A smooth bounded influence function is valuable local protection, but global robustness also depends on the full estimating rule, scale choice, and root selection.

For observations expressed in centered, standardized coordinates, the rejection point iswith value $\infty$ if the set is empty. A finite rejection point means sufficiently distant contamination has zero first-order effect. Boundedness of the influence function does not imply a finite rejection point: the influence function of a quantile has nonzero constant tails. Nor does tending to zero at infinity guarantee a finite rejection point unless those tails become identically zero.

The local-shift sensitivity of a scalar statistical functional isIt measures the sensitivity of the first-order contamination effect to moving the contaminating observation. If the influence function is continuously differentiable with bounded derivative, the mean value theorem gives ${\lambda }^{\ast }\le {\sup }_x|{\mathrm{IF}}^{\prime }(x)|$, and taking $y\to x$ proves equality. A jump in the influence function makes ${\lambda }^{\ast }$ infinite, even when gross-error sensitivity is finite.

Let $q$ be the unique $p$th quantile, with $0<p<1$, and suppose the density $f$ is continuous and positive at $q$. Define quantiles of contaminated distributions using the generalized inverse. For $z\ne q$, differentiating $(1-\epsilon )F(q_{\epsilon })+\epsilon 1_{z\le q_{\epsilon }}=p$ givesAt $z=q$, the generalized-inverse quantile stays exactly $q$ under this contamination, so its derivative is zero. The step formula thus holds $F$-almost everywhere and determines the asymptotic variance $p(1-p)/(nf(q{)}^2)$. Its gross-error sensitivity is $\max (p,1-p)/f(q)$, but its local-shift sensitivity is infinite because of the jump.

This plugs the empirical distribution into the population influence function. For an M-estimator it is $-[n^{-1}{\sum }_i{\partial }_{\theta }\psi (y_i,T(F_n)){]}^{-1}\psi (z,T(F_n))$, when this denominator is well-defined and nonzero. For functionals involving a density at a quantile, an unsmoothed empirical measure may not permit this derivative, so smoothing or an actual sensitivity curve is required.

Add one observation $z$ to the empirical sample, so $F_{n+1,z}=\frac{n}{n+1}{F}_n+\frac{1}{n+1}{\delta }_z$. The displayed finite difference is a finite-sample counterpart of the influence function. Its supremum magnitude measures worst-case one-observation sensitivity; unlike an infinitesimal derivative it records the actual response at contamination fraction $1/(n+1)$.

A one-step estimator starts from a plug-in estimate and adds the empirical mean of an estimated influence function. Under suitable nuisance-estimation and remainder conditions, this correction removes the plug-in estimator's first-order bias and produces an asymptotically linear estimator.

An estimator has an asymptotic linear representation with influence function $\psi$ when

Gross-error sensitivity is the supremum of the absolute influence function over contamination points.

At a distribution with positive density $f$ at its median $m$, the median influence function is $\mathrm{sgn}(x-m)/(2f(m))$ away from $m$.