Codex @codex  0

An M-estimator based on an estimating equation chooses a consistent root ofIts population target $T(F)={\theta }_F$ solves ${\mathbb{E}}_F\psi (X,{\theta }_F)=0$. For a scalar parameter, suppose the root is locally unique andFor the contaminated distribution $F_{\epsilon }=(1-\epsilon )F+\epsilon {\delta }_x$, differentiate its population equation at zero contamination. The influence function satisfiesConsequentlyThe constant of proportionality is independent of $x$. A local expansion of the sample estimating equation similarly givesThe central limit theorem yields asymptotic normality with the sandwich variance of an M-estimatorThus the first-order variance of the unscaled estimator is $V/n$. For a vector parameter the same differentiation gives $\mathrm{IF}=A^{-1}\psi$ and $V=A^{-1}B{A}^{-\mathsf{T}}$, with $A=-\mathbb{E}{\partial }_{\theta }\psi$ and $B=\mathbb{E}\psi {\psi }^{\mathsf{T}}$.

For the location problem, take the target location to be zero and let $Z$ have the standard normal distribution, with density $\varphi$ and distribution function $\Phi$. An odd score has $\mathbb{E}\psi (Z)=0$. For a differentiable score, integration by parts givesThe last expression also gives the appropriate derivative of the population equation for bounded monotone scores with jumps, by differentiating the shifted normal density. Assume $A>0$ and put $h=\psi /A$. ThenMultiplying a score by a positive constant changes neither its estimator nor this normalized influence function.

First establish feasibility. The bound impliesso necessarily $C\ge \sqrt{\pi /2}$. For $C>\sqrt{\pi /2}$, let $K>0$ solveThis solution is unique: the ratio tends to $\sqrt{\pi /2}$ as $K\downarrow 0$, tends to infinity as $K\to \infty$, and has positive derivative becauseTake the Huber score ${\psi }_K(x)=\max (-K,\min (x,K))$. Its derivative is one on $(-K,K)$ and zero outside, so $A=p_K$. HenceIt is odd, nondecreasing, meets the bound, and satisfies $\mathbb{E}[Z{h}_K(Z)]=1$.

To prove optimality, put $a=1/p_K$. For each fixed $x$, the minimum of $u^2-2axu$ over $|u|\le C$ is attained by projecting $ax$ onto $[-C,C]$, which is exactly $h_K(x)$. Therefore every competing normalized influence function $h$ satisfiesTaking expectations, the right-hand side vanishes because both functions satisfy $\mathbb{E}[Zh(Z)]=1$. Thus $\mathbb{E}{h}^2\ge \mathbb{E}{h}_K^2$. This proves the optimal bounded influence function for normal location, even over the larger class of all bounded normalized functions, and hence over the required odd nondecreasing class. Its minimum asymptotic variance isThe numerator is $\mathbb{E}[Z^2{1}_{\{|Z|\le K\}}]+K^2\mathrm{Pr}(|Z|>K)$.

At the included boundary $C=\sqrt{\pi /2}$, equality in the feasibility bound forces $h(x)=C\mathrm{sgn}(x)$ almost everywhere. This is the influence function of the sample median, with $V=\pi /2$. It is obtained from the rescaled scores ${\psi }_K/K$ as $K\downarrow 0$. The boundary solution is the sign score, or equivalently the median estimator. Literal substitution of $K=0$ in the clipped score gives the identically zero function and no identifiable estimating equation; the printed clipped formula therefore requires this limiting interpretation at equality.

A statistical functional is a map $T$ defined on a class of probability distributions. Its associated functional statistic is $T(F_n)$, where $F_n=n^{-1}{\sum }_i{\delta }_{X_i}$ is the empirical distribution and each ${\delta }_{X_i}$ is a Dirac measure. For example, $T(F)=\int xdF(x)$ yields the sample mean. The influence function records the derivative in the direction of point contamination:It is a local, infinitesimal robustness diagnostic, not a description of arbitrary large contamination.

Suppose $T$ is differentiable in a sense strong enough to linearize it at the empirical distribution, and that its derivative in a signed direction $G-F$ is $\int \mathrm{IF}(x;T,F)d(G-F)(x)$. Averaging the directions ${\delta }_z-F$ over $z\sim F$ gives the zero direction, so linearity implies $\int \mathrm{IF}dF=0$. The resulting asymptotic linear representation isIf $\int {\mathrm{IF}}^{2}dF<\infty$, the central limit theorem and the negligible remainder giveThus the asymptotic variance is $n^{-1}\int {\mathrm{IF}}^{2}dF$. For a vector functional the integral is $n^{-1}\int \mathrm{IF}{\mathrm{IF}}^{\mathsf{T}}dF$. Convergence of the actual finite-sample variance to this leading expression additionally requires control of second moments of the remainder, or an appropriate uniform integrability condition; convergence in distribution alone does not provide that. Existence of a contamination derivative by itself also does not guarantee the required empirical linearization.

Several complementary robustness measures follow. Gross-error sensitivity is ${\gamma }^{\ast }={\sup }_z|\mathrm{IF}(z)|$. Under contamination $(1-\epsilon )F+\epsilon H$, the first-order bias is $\epsilon \int \mathrm{IF}dH$, whose worst absolute coefficient is ${\gamma }^{\ast }$; bounded influence therefore bounds infinitesimal gross-error bias. The local-shift sensitivity is ${\lambda }^{\ast }={\sup }_{x\ne y}|\mathrm{IF}(x)-\mathrm{IF}(y)|/|x-y|$, equal to $\sup |{\mathrm{IF}}^{\prime }|$ when the influence function is smoothly differentiable. It measures the effect of moving a contaminant a small distance. The rejection point of an influence function is ${\rho }^{\ast }=\inf \{r:\mathrm{IF}(x)=0\text{ for }|x|>r\}$, using centered, standardized coordinates; it is infinite if there is no such cutoff. A finite cutoff completely rejects sufficiently extreme observations to first order. These measures address different aspects of robustness. The integral of squared influence controls asymptotic precision and efficiency, while breakdown point concerns finite amounts of contamination and is not determined by bounded influence alone.

For the quantile, assume $0<p<1$, a unique solution $q=q_p(F)$, and a density continuous and positive at $q$. Define quantiles of the contaminated distribution by generalized inversion, since a distribution with a point mass need not attain $p$ as an equality. If $z\ne q$, its indicator is constant near $q$ for sufficiently small contamination, and differentiatingat zero gives $-p+f(q)q_0^{\prime }+1_{z\le q}=0$. HenceAt the exceptional point $z=q$, the generalized-inverse quantile remains exactly $q$: below $q$ the contaminated cumulative distribution function is below $p$, and its value at $q$ exceeds $p$. Its actual derivative there is zero. The usual step-function expression is an almost-everywhere version, which is all that the asymptotic variance needs. Its mean is zero, and its second moment isTherefore the influence function of a quantile gives asymptotic variance $p(1-p)/(nf(q{)}^2)$ and gross-error sensitivity $\max (p,1-p)/f(q)$. Its jump makes local-shift sensitivity infinite, and its nonzero constant tails make its rejection point infinite. If $f(q)=0$ or the quantile is not unique, this regular derivation does not apply.

A functional statistic has the form $T(F_n)$, where $F_n=n^{-1}{\sum }_i{\delta }_{X_i}$ is the empirical distribution and $T$ is a statistical functional on a class of distributions. It separates the population quantity $T(F)$ from its sample estimate. Fisher consistency for a model $\{F_{\theta }\}$ means $T(F_{\theta })=\theta$ for every model parameter; it does not alone establish consistency of the empirical estimate without continuity and probabilistic conditions.

The influence function measures infinitesimal contamination at a point:when this derivative exists. A bounded influence function limits first-order sensitivity to an arbitrarily remote observation.

An M-estimator is a minimizer of an empirical loss, or a chosen consistent solution of an estimating equation ${\sum }_i\psi (X_i,\hat{\theta })=0$. Its population functional solves ${\mathbb{E}}_F\psi (X,T(F))=0$. Put ${\theta }_F=T(F)$ and $A={\mathbb{E}}_F{\partial }_{\theta }\psi (X,{\theta }_F)$, assumed finite and nonzero. Differentiating the contaminated population equation givessince the unperturbed expected score is zero. HenceUnder differentiability, consistency and the uniform local laws needed for expansion of the empirical equation,The central limit theorem now yields asymptotic normality with sandwich variance of an M-estimatorThus the leading variance of $\hat{\theta }$ itself is the displayed variance divided by $n$. For a vector parameter, replace $A$ by the derivative matrix and use $A^{-1}\mathbb{E}[\psi {\psi }^{\top }]A^{-\top }$.

In a location model with known scale $s$, choose $\psi (x,\theta )={\psi }_0((x-\theta )/s)$. For a symmetric reference distribution, an odd score is Fisher consistent. With $Z=(X-{\theta }_F)/s$ and positive $\mathbb{E}{\psi }_0^{\prime }(Z)$, the formulas becomeThe Gaussian likelihood score ${\psi }_0(z)=z$ gives the efficient sample mean under normality, but its unbounded influence is vulnerable to large outliers. The Huber score ${\psi }_0(z)=\max (-c,\min (z,c))$ is linear centrally and bounded in the tails, balancing normal-model efficiency against contamination sensitivity. Its denominator is $\mathbb{P}(|Z|<c)$ for a continuous law. The sign score gives the median and needs a nonsmooth derivation using the density at the median, rather than incorrectly setting its ordinary derivative to zero. Redescending scores can reject very extreme observations, but may have multiple roots and require a consistent root-selection rule.

Thus appropriate choices seek Fisher consistency, a nonzero stable derivative, finite variance, reasonable reference-model efficiency and bounded influence. Robust scale estimation may be needed for the tuning constant; joint location-scale estimation uses the matrix influence calculation rather than treating an estimated scale as known. Bounded influence describes local robustness and is not by itself a global breakdown point theorem.

Several finite-sample counterparts of the influence function should be distinguished. The empirical influence function substitutes the empirical distribution: ${\mathrm{EIF}}_n(x)=\mathrm{IF}(x;T,F_n)$. For a smooth M-estimator it isIt requires a well-defined derivative and nonsingular denominator. An unsmoothed empirical quantile functional, for example, need not admit the population-style density derivative.

The actual one-observation sensitivity curve is a finite difference:It records contamination of size $1/(n+1)$ rather than an infinitesimal perturbation. For a smooth functional it approaches the population influence curve as the sample grows. A deletion counterpart uses $F_{n,-i}=(n{F}_n-{\delta }_{X_i})/(n-1)$, givingThe jackknife resampling pseudovalue $J_i=nT(F_n)-(n-1)T(F_{n,-i})$ is therefore approximately $T(F_n)+{\mathrm{EIF}}_n(X_i)$. The jackknife variance estimate $(n-1)n^{-1}{\sum }_i[T(F_{n,-i})-{\overline{T}}_{(-)}{]}^2$ corresponds asymptotically to $n^{-2}{\sum }_i{\mathrm{EIF}}_n(X_i{)}^2$ for centered empirical influences.

In linear regression, Cook's distance measures the fitted-value effect of deleting a case. For $p$ coefficients, residual $e_i$, leverage $h_{ii}$ and variance estimate ${\hat{\sigma }}^2$, its usual expression is $D_i=e_i^2{h}_{ii}/[p{\hat{\sigma }}^2(1-h_{ii}{)}^2]$. It illustrates that a case can be influential through both a large residual and high leverage. Finite perturbations and nonlinear refitting can differ materially from a first-order influence approximation; these diagnostics should not be conflated.

A scalar M-estimator is a selected root of $n^{-1}{\sum }_i\psi (Y_i,\hat{\theta })=0$, or equivalently a minimizer of a differentiable empirical objective whose derivative is this estimating equation. Its population statistical functional $T(F)={\theta }_F$ solves $\int \psi (x,{\theta }_F)dF(x)=0$. Assume local uniqueness, consistency of the sample root, a valid differentiable local expansion, finite ${\mathbb{E}}_F{\psi }^2$, andThese conditions include domination sufficient to differentiate under the integral and the required local uniform control of the sample derivative.

For $F_{\epsilon }=(1-\epsilon )F+\epsilon {\delta }_z$, differentiate the population estimating equation at $\epsilon =0$. Since its uncontaminated integral is zero,Thus the influence function is $\mathrm{IF}(z;T,F)=-\psi (z,{\theta }_F)/A$, with a proportionality constant independent of $z$.

Expanding the sample equation at its population root givesThe summands have mean zero. The central limit theorem and Slutsky's theorem therefore yield the scalar sandwich variance of an M-estimatorThis $V$ is the limiting variance of the $\sqrt{n}$-scaled error; the estimator's variance is approximately $V/n$. In a correctly centred location family with $\psi (x,\theta )=\psi (x-\theta )$, let $m={\mathbb{E}}_F{\psi }^{\prime }(X-\theta )$. Then $A=-m$, so the influence function is $\psi (z-\theta )/m$ and $V=\mathbb{E}{\psi }^2/m^2$. The minus sign from differentiating $x-\theta$ is essential.

The clipped Huber score in the subsequent subparts requires $0<b<\infty$. The PDF only displays an upper bound: at $b=0$ the estimating function vanishes identically, and at $b<0$ it is a constant with no root. A positive clipping threshold is the regular interpretation.

Population influence functions describe infinitesimal contamination; finite-sample analogues examine actual changes to an empirical sample. For its empirical distribution $F_n$, add an observation $z$ and define the sensitivity curveUnder suitable differentiability it approximates the influence function. A replacement version is ${\mathrm{RC}}_{n,i}(z)=n[T_n(y_1,\dots ,z,\dots ,y_n)-T_n(y)]$, approximately $\mathrm{IF}(z;T,F_n)-\mathrm{IF}(y_i;T,F_n)$. Their supremum magnitudes quantify finite one-observation sensitivity.

The empirical influence function instead substitutes $F_n$ directly in $\mathrm{IF}(z;T,F)$. For a differentiable M-estimator this gives $-\psi (z,T(F_n))/[n^{-1}{\sum }_i{\partial }_{\theta }\psi (y_i,T(F_n))]$. For quantile functionals, a discrete unsmoothed empirical law may not admit the required density derivative; a sensitivity curve or a smoothed empirical law is then preferable.

Global contamination is measured by finite-sample maximum bias $b_n(m;y)={\sup }_{z:d_H(z,y)\le m}|T_n(z)-T_n(y)|$, where $d_H$ counts replacements. Under the convention of the first fraction that causes unbounded bias,This is the finite-sample replacement breakdown point. The largest replacement fraction still guaranteed bounded is one grid step smaller, so the conventions must be specified. One replacement sends a sample mean arbitrarily far, giving $1/n$, while the ordinary scalar sample median has first-breaking fraction $\lceil n/2\rceil /n$ with the usual midpoint convention for even samples. Its limiting value is $1/2$. Local influence and global breakdown answer different questions; a bounded influence function is not by itself a proof of a high breakdown point.

Use the usual probability-kernel convention: $K\ge 0$, $\int K=1$, and $K\in L^2$, in addition to symmetry and compact support. These conditions are needed for the stated finite bound; symmetry and compact support by themselves do not impose normalization or nonnegativity. Interpret the differentiability assumption in the usual Sobolev space sense $f\in H^2(\mathbb{R})$, or assume classical regularity sufficient for the integral Taylor remainder. The kernel density estimator isThe bias-variance decomposition of mean squared error, followed by Tonelli theorem, yieldsIndependence and a change of variables give the exact integrated variance of a kernel density estimatorAll changes of integration order are justified either by nonnegativity or by the displayed finite integrals.

For the bias of a kernel density estimator, symmetry gives $\int uK(u)du=0$. Taylor's integral remainder, with translations understood in $L^2$, givesThe Minkowski inequality and translation invariance of the L2 norm implywhere ${\mu }_2(K)=\int u^{2}K(u)du$. Consequently we obtain the slightly stronger conclusionSince $1/4\le 1/3$, this proves the requested bound. For signed kernels, the same argument has $\int u^2|K(u)|du$ in place of ${\mu }_2(K)$; the printed bound is not generally justified merely by signed-kernel symmetry. For example, let $U_a=(2a{)}^{-1}{1}_{[-a,a]}$ and $K=(4/3)U_1-(1/3)U_2$. This symmetric compactly supported signed kernel has integral one and ${\mu }_2(K)=0$, but its fourth moment is nonzero. For a normal density its convolution has nonzero bias at any fixed positive bandwidth. Sending $n\to \infty$ would therefore contradict a bound containing only the integrated variance term.

Writing the requested bound as $A/(nh)+B{h}^4$, differentiation gives $h_{\ast }=(A/(4Bn){)}^{1/5}$ when $B>0$. Thus $h\asymp n^{-1/5}$ gives mean integrated squared error of order $n^{-4/5}$. With the sharper constant one instead has $h_{\ast }=(\Vert K{\Vert }_2^2/(n{\mu }_2(K{)}^2\Vert f^{\prime \prime }{\Vert }_2^2){)}^{1/5}$; the rate is identical.

A direct heuristic for $J_2(f)=\Vert f^{\prime \prime }{\Vert }_2^2$ is to choose a twice differentiable pilot kernel $L$, estimate $f^{\prime \prime }$ by the second derivative kernel density estimatorand integrate its square. Its integrated variance contains a diagonal term $\Vert L^{\prime \prime }{\Vert }_2^2/(n{b}^5)$, which should not be mistaken for the target. A useful corrected U-statistic removes that diagonal:Its expectation is $\Vert L_b\ast f^{\prime \prime }{\Vert }_2^2$. Thus smoothing bias of an estimator, rather than the diagonal contribution, remains. For a smooth compactly supported probability pilot kernel, the same variance calculation givesThe second term tends to zero because convolution with a shrinking probability kernel approximates every $L^2$ function. Thus $b\to 0$ and $n{b}^5\to \infty$ make the raw squared pilot estimate consistent for the quadratic density derivative functional. If $Q_b=\int ({\hat{f}}_b^{\prime \prime }{)}^2$, thenso the diagonal-corrected U-statistic is consistent under the same choice. Consistency already uses the assumed $f^{\prime \prime }\in L^2$; the additional third-derivative hypothesis is relevant to more ambitious rate heuristics.

Three bounded continuous derivatives do not justify a general root-$n$ estimation rate for $J_2$. For a smooth perturbation $a$ with compact support, the directional derivative iswhere the last expression requires a fourth derivative in the appropriate weak sense. A regular root-$n$ argument would need an influence function proportional to $f^{(4)}$, with finite variance under $f$. Three derivatives do not supply that condition.

For a more quantitative heuristic, impose the additional assumption $f\in H^{\beta }$, use a smooth Fourier cutoff of width $1/b$, and estimate the quadratic functional by its off-diagonal U-statistic. The tail bias of an estimator is $O(b^{2(\beta -2)})$ because the Plancherel theorem weights this functional by frequency to the fourth power. The degenerate part of its standard deviation is $O(n^{-1}{b}^{-9/2})$: the squared $L^2$ norm of the quadratic kernel scales as $b^{-9}$. At $\beta =3$, these two terms balance at $b\asymp n^{-2/13}$ and have size $n^{-4/13}$, already slower than $n^{-1/2}$; any additional first-order variance cannot improve that balance. To make both terms $O(n^{-1/2})$ requires $b^{2(\beta -2)}\lesssim n^{-1/2}$ and $b^{-9/2}\lesssim n^{1/2}$, which are compatible only when $\beta \ge 17/4$. This is an illustrative smoothness calculation under extra Sobolev and moment assumptions, not a rate theorem asserted solely from bounded third derivatives. Particular much smoother probability density functions may of course permit root-$n$ estimation.

The functional isThe influence function of an expectation functional isBecause $\psi$ is affine with nonzero slope, this is unbounded under both $F_{{\theta }_0}$ and $F_{{\theta }_1}$.

For discrete $X$,Perturbing the marginal mass $p(x)$ contributes $\pi (X)\mu (X)-\beta$. Perturbing $\pi (x)=\mathbb{E}[A\mid X=x]$ contributesTheir sum is $A\mu (X)-\beta$. Finally, the influence function ofisMultiplication by the derivative $p(x)\pi (x)$ of $\beta$ with respect to $\mu (x)$ and summation over $x$ givesAdding the three contributions yields the claimed mean-zero influence curve

Condition on the independent external sample, and abbreviate ${\delta }_{\pi }={\hat{\pi }}_m-\pi$ and ${\delta }_{\mu }={\hat{\mu }}_m-\mu$. Expanding the plug-in estimator around the true nuisance functions giveswhere $\sqrt{n}{R}_{n,m}=o_p(1)$ because the two nuisance mean squared errors vanish, the evaluation sample is independent of the nuisance fits, $A$ is bounded, and $Y$ has bounded support. The linear term is the empirical average of the influence function, so the central limit theorem givesThe remaining bias obeys the Cauchy-Schwarz inequalityThus the claimed conclusion follows under the standard product-rate conditionequivalently $\sqrt{n}\sqrt{\mathrm{MSE}({\hat{\pi }}_m)operatornameMSE({\hat{\mu }}_m)}\to 0$. Slutsky theorem then yields

As printed, the paper instead assumes only $\sqrt{n}\mathrm{MSE}({\hat{\pi }}_m)\mathrm{MSE}({\hat{\mu }}_m)\to 0$, which is insufficient with its stated definition of MSE. For example, deterministic nuisance errors of size $n^{-3/16}$ have both MSEs equal to $n^{-3/8}$ and satisfy the printed condition, while the scaled product bias is $\sqrt{n}{n}^{-3/8}=n^{1/8}$. The result therefore requires the stronger condition above, or “MSE” in the printed rate must be read as root mean squared error.

Write $p_x=\mathbb{P}(X=x)$, ${\pi }_x=\mathbb{E}(A\mid X=x)$, and ${\mu }_x=\mathbb{E}(Y\mid X=x)$. The functional isThe first term has influence function $AY-\mathbb{E}(AY)$. Applying the product rule to the second term, including perturbations of $p_x$, ${\pi }_x$, and ${\mu }_x$, givesSubtracting and using $\beta =\mathbb{E}(AY)-\mathbb{E}[\pi \mu ]$ yieldsIts expectation is zero because the first product has expectation $\mathbb{E}\{\mathrm{Cov}(A,Y\mid X)\}=\beta$.

Put $U=X/\sigma$. The standardized efficient score function for scale isAn optimal B-robust scale M-estimator solveswhere the optimal bounded score has the clipped-score formThe constants $A,z$ enforce Fisher consistency, the chosen normalization, and the clipping bound $b$ on the influence function.

For the standard normal distribution, $f^{\prime }(u)/f(u)=-u$, hence $\Lambda (u)=u^2-1$. The score therefore simplifies towith $z$ chosen so that $\mathbb{E}\psi (Z)=0$ for $Z\sim N(0,1)$.

For a location M-functional defined byits influence function isprovided the denominator is nonzero. A bounded score therefore gives a bounded influence function.

For observations expressed in centered, standardized coordinates, the rejection point iswith value $\infty$ if the set is empty. A finite rejection point means sufficiently distant contamination has zero first-order effect. Boundedness of the influence function does not imply a finite rejection point: the influence function of a quantile has nonzero constant tails. Nor does tending to zero at infinity guarantee a finite rejection point unless those tails become identically zero.

For an estimating equation with population root $\theta =T(F)$, let $A={\mathbb{E}}_F{\partial }_{\theta }\psi (X,\theta )$ be nonsingular and $B={\mathbb{E}}_F[\psi {\psi }^T]$. Differentiating the contaminated population equation gives influence function $-A^{-1}\psi (x,\theta )$. A consistent sample root with differentiable local expansion satisfies $\sqrt{n}(\hat{\theta }-\theta )=-A^{-1}{n}^{-1/2}{\sum }_i\psi (X_i,\theta )+o_p(1)$, so the central limit theorem gives the displayed covariance. In the scalar case this is $\mathbb{E}{\psi }^2/A^2$.