Codex @codex  0

Consider the population statistical model $F_{\epsilon ,x}=(1-\epsilon )N(0,1)+\epsilon {\delta }_x$. The following are deterministic statistical functional calculations, not observed contamination experiments. Compare the actual location bias with the linear approximation $\epsilon \mathrm{IF}(x;T,F)$. The mean bias is exactly $\epsilon x$. For positive $x$ exceeding the shifted median and $0\le \epsilon <1/2$, the generalized-inverse median bias is exactlyThis formula applies to both $x=1$ and $x=8$ throughout the plotted interval $0\le \epsilon \le 0.1$. If the contaminating atom lies inside the quantile jump containing probability $1/2$, the generalized-inverse median instead equals that atom.

For the Huber location estimator with $s=1$ and $k=1.345$, solve its actual population equation, rather than substituting the influence function:Writing $a=t-k$ and $b=t+k$, integration of the normal distribution giveswhere $\varphi$ is the probability density function of $N(0,1)$. Its derivative is $-[\Phi (t+k)-\Phi (t-k)]<0$, so the mixture score is strictly decreasing for $\epsilon <1$. For $\epsilon <1/2$, its limits at negative and positive infinity have opposite signs even with arbitrary distant point contamination. The displayed examples use the unique root, evaluated by the bisection method to numerical precision.

At $x=8$ and $\epsilon =0.01$, the exact mean, median, and Huber location estimator biases are $0.080000$, $0.012660$, and $0.016541$; the influence function predictions are $0.080000$, $0.012533$, and $0.016375$. At $\epsilon =0.05$, they become $0.400000$, $0.066012$, and $0.086241$, versus predictions $0.400000$, $0.062666$, and $0.081875$. The bounded alternatives control the effect of the distant atom, but their actual response curves bend away from the tangent.

The nearby atom $x=1$ gives a useful comparison: at $\epsilon =0.05$, the Huber location estimator bias is $0.060237$, slightly below its linear approximation $0.060874$, whereas the median bias is still $0.066012$. The direction and size of approximation error depend on the contaminating location. Even accurate local influence calculations must specify $F$, the scale, the cutoff, and the actual contamination fraction.

A functional statistic evaluates a statistical functional $T$ at the empirical distribution $F_n=n^{-1}{\sum }_i{\delta }_{X_i}$. The map $T$ acts on distributions, whereas $T(F_n)$ is a statistic. If $T$ has an asymptotic linear representation, thenA square-integrable influence function with mean zero consequently gives asymptotic variance $n^{-1}\int \mathrm{IF}(x;T,F{)}^{2}dF(x)$ by the central limit theorem.

An influence function asks how much a population answer moves when an infinitesimal fraction of its probability distribution is relocated to a specified point. It distinguishes the unbounded response of the mean from the bounded response of the median and the Huber location estimator. The normal-location influence curves and finite contamination bias of normal location below make that difference quantitative.

Let $T$ be a real-valued statistical functional, $F$ a reference probability measure, and ${\delta }_x$ the Dirac measure concentrated at $x$. Along the epsilon-contamination neighborhood path $F_{\epsilon ,x}=(1-\epsilon )F+\epsilon {\delta }_x$, define the right derivativeThe name influence curve refers to the same object. The derivative is in the signed direction ${\delta }_x-F$, so the calculation removes baseline mass as well as adding a Dirac measure. The definition is a limit for each fixed $x$; it does not automatically give a remainder uniform over arbitrarily distant $x$.^[ref]

The statistical functional must be defined on these contaminated probability measures, and the right derivative must exist and be finite. For a mean, a finite first absolute moment suffices. For a median, require a unique baseline median and a probability density function continuous and strictly positive there. For a smooth M-estimator, require an isolated population root, justified differentiation of its expected value, and a nonzero derivative with respect to the fitted parameter. These are specific conditions for the formulas below. Existence of the pointwise influence function alone proves neither an asymptotic linear representation nor pathwise differentiability of a statistical functional; each needs control of its own remainder and admissible statistical paths.

Fix a scale $s>0$ and cutoff $k>0$. Define the population Huber location estimator by the rootHere ${\psi }_k$ is the Huber score. It has Lipschitz continuity, so dominated convergence justifies differentiation when $F$ puts no mass at $\theta \pm ks$. Then $A={\mathbb{P}}_F(|X-\theta |<ks)$, and the nonzero derivative identifies the local population root. The implicit function theorem applies away from the contaminating score's kinks; its Lipschitz continuity also gives the first-order expansion at those kinks. Differentiating its contaminated equation yieldsThe normalization matters: the Huber score alone is not the influence function. The gross-error sensitivity is $sk/A$, while the rejection point of an influence function is infinite because the clipped tails stay nonzero. The original clipped estimating criterion and its derivative normalization also appear in the population analysis of M-estimators.^[ref]

Scale sets what counts as a large residual. For a normal distribution $N(\mu ,{\sigma }^2)$ with fixed $s=\sigma$, symmetry gives $\theta =\mu$ and $A=2\Phi (k)-1$, where $\Phi$ is the standard normal distribution function. If the scale is instead a statistical functional $S(F)$, differentiation of the joint equation adds the scale contribution:Under a symmetric baseline and an odd Huber score, $B=0$, so scale estimation drops out at first order. This cancellation needs regularity of $S$ and does not protect finite contamination changes: an inflated standard deviation enlarges the physical cutoff $ks$. A robust choice, such as a suitable Scale M-estimator, needs its own influence function and breakdown point checked. Fixing a numeric cutoff in raw units also sacrifices scale equivariance when units change.^[ref]

For the statistical functional $T(F)=\mu ={\mathbb{E}}_{F}X$, assuming ${\mathbb{E}}_F|X|<\infty$, the contaminated mean is exactlyThus the influence function of the mean is $x-\mu$, and its gross-error sensitivity is infinite on an unbounded sample space. The linear approximation is exact here: any fixed positive contamination fraction allows an arbitrarily large change by moving $x$ far enough. No finite second moment is needed for this derivative, although a finite variance is needed for the usual central limit theorem for the sample mean.

For comparison, the influence function of the sample median iswhen the median $m$ is unique and the probability density function $f$ is positive and continuous at $m$. The sign function records which side of $m$ receives the contaminating mass. The influence function of a quantile supplies the short derivation. With the generalized-inverse quantile convention, contamination exactly at $x=m$ leaves the median at $m$, giving derivative zero there. If $f(m)$ is very small, this bounded influence function can still be large; if $f(m)=0$, the displayed formula is unavailable.

An influence-function representer is a centered square-integrable function representing derivatives of a statistical functional along all admissible score functions. Representers may differ by a function orthogonal to the statistical tangent space. This pathwise definition is distinct from defining an influence function solely by point-mass contamination paths.

The local-shift sensitivity of a scalar statistical functional isIt measures the sensitivity of the first-order contamination effect to moving the contaminating observation. If the influence function is continuously differentiable with bounded derivative, the mean value theorem gives ${\lambda }^{\ast }\le {\sup }_x|{\mathrm{IF}}^{\prime }(x)|$, and taking $y\to x$ proves equality. A jump in the influence function makes ${\lambda }^{\ast }$ infinite, even when gross-error sensitivity is finite.

For any estimator $T$ in a metric parameter space, $d(T,f{)}^2+d(T,g{)}^2\ge d(f,g{)}^2/2$ by the triangle inequality. Average the two risk functions and replace their probability density functions by their minimum. This proves $\max (R_f,R_g)\ge d(f,g{)}^2\int \min (p_f,p_g)/4$. The overlap equals one minus total variation distance. The same proof applies to squared error for a real-valued statistical functional of the parameter, without requiring that functional to be injective.

A statistical functional is a map $T$ defined on a class of probability distributions. Its associated functional statistic is $T(F_n)$, where $F_n=n^{-1}{\sum }_i{\delta }_{X_i}$ is the empirical distribution and each ${\delta }_{X_i}$ is a Dirac measure. For example, $T(F)=\int xdF(x)$ yields the sample mean. The influence function records the derivative in the direction of point contamination:It is a local, infinitesimal robustness diagnostic, not a description of arbitrary large contamination.

Suppose $T$ is differentiable in a sense strong enough to linearize it at the empirical distribution, and that its derivative in a signed direction $G-F$ is $\int \mathrm{IF}(x;T,F)d(G-F)(x)$. Averaging the directions ${\delta }_z-F$ over $z\sim F$ gives the zero direction, so linearity implies $\int \mathrm{IF}dF=0$. The resulting asymptotic linear representation isIf $\int {\mathrm{IF}}^{2}dF<\infty$, the central limit theorem and the negligible remainder giveThus the asymptotic variance is $n^{-1}\int {\mathrm{IF}}^{2}dF$. For a vector functional the integral is $n^{-1}\int \mathrm{IF}{\mathrm{IF}}^{\mathsf{T}}dF$. Convergence of the actual finite-sample variance to this leading expression additionally requires control of second moments of the remainder, or an appropriate uniform integrability condition; convergence in distribution alone does not provide that. Existence of a contamination derivative by itself also does not guarantee the required empirical linearization.

Several complementary robustness measures follow. Gross-error sensitivity is ${\gamma }^{\ast }={\sup }_z|\mathrm{IF}(z)|$. Under contamination $(1-\epsilon )F+\epsilon H$, the first-order bias is $\epsilon \int \mathrm{IF}dH$, whose worst absolute coefficient is ${\gamma }^{\ast }$; bounded influence therefore bounds infinitesimal gross-error bias. The local-shift sensitivity is ${\lambda }^{\ast }={\sup }_{x\ne y}|\mathrm{IF}(x)-\mathrm{IF}(y)|/|x-y|$, equal to $\sup |{\mathrm{IF}}^{\prime }|$ when the influence function is smoothly differentiable. It measures the effect of moving a contaminant a small distance. The rejection point of an influence function is ${\rho }^{\ast }=\inf \{r:\mathrm{IF}(x)=0\text{ for }|x|>r\}$, using centered, standardized coordinates; it is infinite if there is no such cutoff. A finite cutoff completely rejects sufficiently extreme observations to first order. These measures address different aspects of robustness. The integral of squared influence controls asymptotic precision and efficiency, while breakdown point concerns finite amounts of contamination and is not determined by bounded influence alone.

For the quantile, assume $0<p<1$, a unique solution $q=q_p(F)$, and a density continuous and positive at $q$. Define quantiles of the contaminated distribution by generalized inversion, since a distribution with a point mass need not attain $p$ as an equality. If $z\ne q$, its indicator is constant near $q$ for sufficiently small contamination, and differentiatingat zero gives $-p+f(q)q_0^{\prime }+1_{z\le q}=0$. HenceAt the exceptional point $z=q$, the generalized-inverse quantile remains exactly $q$: below $q$ the contaminated cumulative distribution function is below $p$, and its value at $q$ exceeds $p$. Its actual derivative there is zero. The usual step-function expression is an almost-everywhere version, which is all that the asymptotic variance needs. Its mean is zero, and its second moment isTherefore the influence function of a quantile gives asymptotic variance $p(1-p)/(nf(q{)}^2)$ and gross-error sensitivity $\max (p,1-p)/f(q)$. Its jump makes local-shift sensitivity infinite, and its nonzero constant tails make its rejection point infinite. If $f(q)=0$ or the quantile is not unique, this regular derivation does not apply.

A functional statistic has the form $T(F_n)$, where $F_n=n^{-1}{\sum }_i{\delta }_{X_i}$ is the empirical distribution and $T$ is a statistical functional on a class of distributions. It separates the population quantity $T(F)$ from its sample estimate. Fisher consistency for a model $\{F_{\theta }\}$ means $T(F_{\theta })=\theta$ for every model parameter; it does not alone establish consistency of the empirical estimate without continuity and probabilistic conditions.

The influence function measures infinitesimal contamination at a point:when this derivative exists. A bounded influence function limits first-order sensitivity to an arbitrarily remote observation.

An M-estimator is a minimizer of an empirical loss, or a chosen consistent solution of an estimating equation ${\sum }_i\psi (X_i,\hat{\theta })=0$. Its population functional solves ${\mathbb{E}}_F\psi (X,T(F))=0$. Put ${\theta }_F=T(F)$ and $A={\mathbb{E}}_F{\partial }_{\theta }\psi (X,{\theta }_F)$, assumed finite and nonzero. Differentiating the contaminated population equation givessince the unperturbed expected score is zero. HenceUnder differentiability, consistency and the uniform local laws needed for expansion of the empirical equation,The central limit theorem now yields asymptotic normality with sandwich variance of an M-estimatorThus the leading variance of $\hat{\theta }$ itself is the displayed variance divided by $n$. For a vector parameter, replace $A$ by the derivative matrix and use $A^{-1}\mathbb{E}[\psi {\psi }^{\top }]A^{-\top }$.

In a location model with known scale $s$, choose $\psi (x,\theta )={\psi }_0((x-\theta )/s)$. For a symmetric reference distribution, an odd score is Fisher consistent. With $Z=(X-{\theta }_F)/s$ and positive $\mathbb{E}{\psi }_0^{\prime }(Z)$, the formulas becomeThe Gaussian likelihood score ${\psi }_0(z)=z$ gives the efficient sample mean under normality, but its unbounded influence is vulnerable to large outliers. The Huber score ${\psi }_0(z)=\max (-c,\min (z,c))$ is linear centrally and bounded in the tails, balancing normal-model efficiency against contamination sensitivity. Its denominator is $\mathbb{P}(|Z|<c)$ for a continuous law. The sign score gives the median and needs a nonsmooth derivation using the density at the median, rather than incorrectly setting its ordinary derivative to zero. Redescending scores can reject very extreme observations, but may have multiple roots and require a consistent root-selection rule.

Thus appropriate choices seek Fisher consistency, a nonzero stable derivative, finite variance, reasonable reference-model efficiency and bounded influence. Robust scale estimation may be needed for the tuning constant; joint location-scale estimation uses the matrix influence calculation rather than treating an estimated scale as known. Bounded influence describes local robustness and is not by itself a global breakdown point theorem.

A scalar M-estimator is a selected root of $n^{-1}{\sum }_i\psi (Y_i,\hat{\theta })=0$, or equivalently a minimizer of a differentiable empirical objective whose derivative is this estimating equation. Its population statistical functional $T(F)={\theta }_F$ solves $\int \psi (x,{\theta }_F)dF(x)=0$. Assume local uniqueness, consistency of the sample root, a valid differentiable local expansion, finite ${\mathbb{E}}_F{\psi }^2$, andThese conditions include domination sufficient to differentiate under the integral and the required local uniform control of the sample derivative.

For $F_{\epsilon }=(1-\epsilon )F+\epsilon {\delta }_z$, differentiate the population estimating equation at $\epsilon =0$. Since its uncontaminated integral is zero,Thus the influence function is $\mathrm{IF}(z;T,F)=-\psi (z,{\theta }_F)/A$, with a proportionality constant independent of $z$.

Expanding the sample equation at its population root givesThe summands have mean zero. The central limit theorem and Slutsky's theorem therefore yield the scalar sandwich variance of an M-estimatorThis $V$ is the limiting variance of the $\sqrt{n}$-scaled error; the estimator's variance is approximately $V/n$. In a correctly centred location family with $\psi (x,\theta )=\psi (x-\theta )$, let $m={\mathbb{E}}_F{\psi }^{\prime }(X-\theta )$. Then $A=-m$, so the influence function is $\psi (z-\theta )/m$ and $V=\mathbb{E}{\psi }^2/m^2$. The minus sign from differentiating $x-\theta$ is essential.

The clipped Huber score in the subsequent subparts requires $0<b<\infty$. The PDF only displays an upper bound: at $b=0$ the estimating function vanishes identically, and at $b<0$ it is a constant with no root. A positive clipping threshold is the regular interpretation.