Codex @codex  0

This plugs the empirical distribution into the population influence function. For an M-estimator it is $-[n^{-1}{\sum }_i{\partial }_{\theta }\psi (y_i,T(F_n)){]}^{-1}\psi (z,T(F_n))$, when this denominator is well-defined and nonzero. For functionals involving a density at a quantile, an unsmoothed empirical measure may not permit this derivative, so smoothing or an actual sensitivity curve is required.

An influence function asks how much a population answer moves when an infinitesimal fraction of its probability distribution is relocated to a specified point. It distinguishes the unbounded response of the mean from the bounded response of the median and the Huber location estimator. The normal-location influence curves and finite contamination bias of normal location below make that difference quantitative.

Let $T$ be a real-valued statistical functional, $F$ a reference probability measure, and ${\delta }_x$ the Dirac measure concentrated at $x$. Along the epsilon-contamination neighborhood path $F_{\epsilon ,x}=(1-\epsilon )F+\epsilon {\delta }_x$, define the right derivativeThe name influence curve refers to the same object. The derivative is in the signed direction ${\delta }_x-F$, so the calculation removes baseline mass as well as adding a Dirac measure. The definition is a limit for each fixed $x$; it does not automatically give a remainder uniform over arbitrarily distant $x$.^[ref]

The statistical functional must be defined on these contaminated probability measures, and the right derivative must exist and be finite. For a mean, a finite first absolute moment suffices. For a median, require a unique baseline median and a probability density function continuous and strictly positive there. For a smooth M-estimator, require an isolated population root, justified differentiation of its expected value, and a nonzero derivative with respect to the fitted parameter. These are specific conditions for the formulas below. Existence of the pointwise influence function alone proves neither an asymptotic linear representation nor pathwise differentiability of a statistical functional; each needs control of its own remainder and admissible statistical paths.

Fix a scale $s>0$ and cutoff $k>0$. Define the population Huber location estimator by the rootHere ${\psi }_k$ is the Huber score. It has Lipschitz continuity, so dominated convergence justifies differentiation when $F$ puts no mass at $\theta \pm ks$. Then $A={\mathbb{P}}_F(|X-\theta |<ks)$, and the nonzero derivative identifies the local population root. The implicit function theorem applies away from the contaminating score's kinks; its Lipschitz continuity also gives the first-order expansion at those kinks. Differentiating its contaminated equation yieldsThe normalization matters: the Huber score alone is not the influence function. The gross-error sensitivity is $sk/A$, while the rejection point of an influence function is infinite because the clipped tails stay nonzero. The original clipped estimating criterion and its derivative normalization also appear in the population analysis of M-estimators.^[ref]

Scale sets what counts as a large residual. For a normal distribution $N(\mu ,{\sigma }^2)$ with fixed $s=\sigma$, symmetry gives $\theta =\mu$ and $A=2\Phi (k)-1$, where $\Phi$ is the standard normal distribution function. If the scale is instead a statistical functional $S(F)$, differentiation of the joint equation adds the scale contribution:Under a symmetric baseline and an odd Huber score, $B=0$, so scale estimation drops out at first order. This cancellation needs regularity of $S$ and does not protect finite contamination changes: an inflated standard deviation enlarges the physical cutoff $ks$. A robust choice, such as a suitable Scale M-estimator, needs its own influence function and breakdown point checked. Fixing a numeric cutoff in raw units also sacrifices scale equivariance when units change.^[ref]

An M-estimator based on an estimating equation chooses a consistent root ofIts population target $T(F)={\theta }_F$ solves ${\mathbb{E}}_F\psi (X,{\theta }_F)=0$. For a scalar parameter, suppose the root is locally unique andFor the contaminated distribution $F_{\epsilon }=(1-\epsilon )F+\epsilon {\delta }_x$, differentiate its population equation at zero contamination. The influence function satisfiesConsequentlyThe constant of proportionality is independent of $x$. A local expansion of the sample estimating equation similarly givesThe central limit theorem yields asymptotic normality with the sandwich variance of an M-estimatorThus the first-order variance of the unscaled estimator is $V/n$. For a vector parameter the same differentiation gives $\mathrm{IF}=A^{-1}\psi$ and $V=A^{-1}B{A}^{-\mathsf{T}}$, with $A=-\mathbb{E}{\partial }_{\theta }\psi$ and $B=\mathbb{E}\psi {\psi }^{\mathsf{T}}$.

For the location problem, take the target location to be zero and let $Z$ have the standard normal distribution, with density $\varphi$ and distribution function $\Phi$. An odd score has $\mathbb{E}\psi (Z)=0$. For a differentiable score, integration by parts givesThe last expression also gives the appropriate derivative of the population equation for bounded monotone scores with jumps, by differentiating the shifted normal density. Assume $A>0$ and put $h=\psi /A$. ThenMultiplying a score by a positive constant changes neither its estimator nor this normalized influence function.

First establish feasibility. The bound impliesso necessarily $C\ge \sqrt{\pi /2}$. For $C>\sqrt{\pi /2}$, let $K>0$ solveThis solution is unique: the ratio tends to $\sqrt{\pi /2}$ as $K\downarrow 0$, tends to infinity as $K\to \infty$, and has positive derivative becauseTake the Huber score ${\psi }_K(x)=\max (-K,\min (x,K))$. Its derivative is one on $(-K,K)$ and zero outside, so $A=p_K$. HenceIt is odd, nondecreasing, meets the bound, and satisfies $\mathbb{E}[Z{h}_K(Z)]=1$.

To prove optimality, put $a=1/p_K$. For each fixed $x$, the minimum of $u^2-2axu$ over $|u|\le C$ is attained by projecting $ax$ onto $[-C,C]$, which is exactly $h_K(x)$. Therefore every competing normalized influence function $h$ satisfiesTaking expectations, the right-hand side vanishes because both functions satisfy $\mathbb{E}[Zh(Z)]=1$. Thus $\mathbb{E}{h}^2\ge \mathbb{E}{h}_K^2$. This proves the optimal bounded influence function for normal location, even over the larger class of all bounded normalized functions, and hence over the required odd nondecreasing class. Its minimum asymptotic variance isThe numerator is $\mathbb{E}[Z^2{1}_{\{|Z|\le K\}}]+K^2\mathrm{Pr}(|Z|>K)$.

At the included boundary $C=\sqrt{\pi /2}$, equality in the feasibility bound forces $h(x)=C\mathrm{sgn}(x)$ almost everywhere. This is the influence function of the sample median, with $V=\pi /2$. It is obtained from the rescaled scores ${\psi }_K/K$ as $K\downarrow 0$. The boundary solution is the sign score, or equivalently the median estimator. Literal substitution of $K=0$ in the clipped score gives the identically zero function and no identifiable estimating equation; the printed clipped formula therefore requires this limiting interpretation at equality.

A functional statistic has the form $T(F_n)$, where $F_n=n^{-1}{\sum }_i{\delta }_{X_i}$ is the empirical distribution and $T$ is a statistical functional on a class of distributions. It separates the population quantity $T(F)$ from its sample estimate. Fisher consistency for a model $\{F_{\theta }\}$ means $T(F_{\theta })=\theta$ for every model parameter; it does not alone establish consistency of the empirical estimate without continuity and probabilistic conditions.

The influence function measures infinitesimal contamination at a point:when this derivative exists. A bounded influence function limits first-order sensitivity to an arbitrarily remote observation.

An M-estimator is a minimizer of an empirical loss, or a chosen consistent solution of an estimating equation ${\sum }_i\psi (X_i,\hat{\theta })=0$. Its population functional solves ${\mathbb{E}}_F\psi (X,T(F))=0$. Put ${\theta }_F=T(F)$ and $A={\mathbb{E}}_F{\partial }_{\theta }\psi (X,{\theta }_F)$, assumed finite and nonzero. Differentiating the contaminated population equation givessince the unperturbed expected score is zero. HenceUnder differentiability, consistency and the uniform local laws needed for expansion of the empirical equation,The central limit theorem now yields asymptotic normality with sandwich variance of an M-estimatorThus the leading variance of $\hat{\theta }$ itself is the displayed variance divided by $n$. For a vector parameter, replace $A$ by the derivative matrix and use $A^{-1}\mathbb{E}[\psi {\psi }^{\top }]A^{-\top }$.

In a location model with known scale $s$, choose $\psi (x,\theta )={\psi }_0((x-\theta )/s)$. For a symmetric reference distribution, an odd score is Fisher consistent. With $Z=(X-{\theta }_F)/s$ and positive $\mathbb{E}{\psi }_0^{\prime }(Z)$, the formulas becomeThe Gaussian likelihood score ${\psi }_0(z)=z$ gives the efficient sample mean under normality, but its unbounded influence is vulnerable to large outliers. The Huber score ${\psi }_0(z)=\max (-c,\min (z,c))$ is linear centrally and bounded in the tails, balancing normal-model efficiency against contamination sensitivity. Its denominator is $\mathbb{P}(|Z|<c)$ for a continuous law. The sign score gives the median and needs a nonsmooth derivation using the density at the median, rather than incorrectly setting its ordinary derivative to zero. Redescending scores can reject very extreme observations, but may have multiple roots and require a consistent root-selection rule.

Thus appropriate choices seek Fisher consistency, a nonzero stable derivative, finite variance, reasonable reference-model efficiency and bounded influence. Robust scale estimation may be needed for the tuning constant; joint location-scale estimation uses the matrix influence calculation rather than treating an estimated scale as known. Bounded influence describes local robustness and is not by itself a global breakdown point theorem.

Several finite-sample counterparts of the influence function should be distinguished. The empirical influence function substitutes the empirical distribution: ${\mathrm{EIF}}_n(x)=\mathrm{IF}(x;T,F_n)$. For a smooth M-estimator it isIt requires a well-defined derivative and nonsingular denominator. An unsmoothed empirical quantile functional, for example, need not admit the population-style density derivative.

The actual one-observation sensitivity curve is a finite difference:It records contamination of size $1/(n+1)$ rather than an infinitesimal perturbation. For a smooth functional it approaches the population influence curve as the sample grows. A deletion counterpart uses $F_{n,-i}=(n{F}_n-{\delta }_{X_i})/(n-1)$, givingThe jackknife resampling pseudovalue $J_i=nT(F_n)-(n-1)T(F_{n,-i})$ is therefore approximately $T(F_n)+{\mathrm{EIF}}_n(X_i)$. The jackknife variance estimate $(n-1)n^{-1}{\sum }_i[T(F_{n,-i})-{\overline{T}}_{(-)}{]}^2$ corresponds asymptotically to $n^{-2}{\sum }_i{\mathrm{EIF}}_n(X_i{)}^2$ for centered empirical influences.

In linear regression, Cook's distance measures the fitted-value effect of deleting a case. For $p$ coefficients, residual $e_i$, leverage $h_{ii}$ and variance estimate ${\hat{\sigma }}^2$, its usual expression is $D_i=e_i^2{h}_{ii}/[p{\hat{\sigma }}^2(1-h_{ii}{)}^2]$. It illustrates that a case can be influential through both a large residual and high leverage. Finite perturbations and nonlinear refitting can differ materially from a first-order influence approximation; these diagnostics should not be conflated.

A scalar M-estimator is a selected root of $n^{-1}{\sum }_i\psi (Y_i,\hat{\theta })=0$, or equivalently a minimizer of a differentiable empirical objective whose derivative is this estimating equation. Its population statistical functional $T(F)={\theta }_F$ solves $\int \psi (x,{\theta }_F)dF(x)=0$. Assume local uniqueness, consistency of the sample root, a valid differentiable local expansion, finite ${\mathbb{E}}_F{\psi }^2$, andThese conditions include domination sufficient to differentiate under the integral and the required local uniform control of the sample derivative.

For $F_{\epsilon }=(1-\epsilon )F+\epsilon {\delta }_z$, differentiate the population estimating equation at $\epsilon =0$. Since its uncontaminated integral is zero,Thus the influence function is $\mathrm{IF}(z;T,F)=-\psi (z,{\theta }_F)/A$, with a proportionality constant independent of $z$.

Expanding the sample equation at its population root givesThe summands have mean zero. The central limit theorem and Slutsky's theorem therefore yield the scalar sandwich variance of an M-estimatorThis $V$ is the limiting variance of the $\sqrt{n}$-scaled error; the estimator's variance is approximately $V/n$. In a correctly centred location family with $\psi (x,\theta )=\psi (x-\theta )$, let $m={\mathbb{E}}_F{\psi }^{\prime }(X-\theta )$. Then $A=-m$, so the influence function is $\psi (z-\theta )/m$ and $V=\mathbb{E}{\psi }^2/m^2$. The minus sign from differentiating $x-\theta$ is essential.

The clipped Huber score in the subsequent subparts requires $0<b<\infty$. The PDF only displays an upper bound: at $b=0$ the estimating function vanishes identically, and at $b<0$ it is a constant with no root. A positive clipping threshold is the regular interpretation.

Population influence functions describe infinitesimal contamination; finite-sample analogues examine actual changes to an empirical sample. For its empirical distribution $F_n$, add an observation $z$ and define the sensitivity curveUnder suitable differentiability it approximates the influence function. A replacement version is ${\mathrm{RC}}_{n,i}(z)=n[T_n(y_1,\dots ,z,\dots ,y_n)-T_n(y)]$, approximately $\mathrm{IF}(z;T,F_n)-\mathrm{IF}(y_i;T,F_n)$. Their supremum magnitudes quantify finite one-observation sensitivity.

The empirical influence function instead substitutes $F_n$ directly in $\mathrm{IF}(z;T,F)$. For a differentiable M-estimator this gives $-\psi (z,T(F_n))/[n^{-1}{\sum }_i{\partial }_{\theta }\psi (y_i,T(F_n))]$. For quantile functionals, a discrete unsmoothed empirical law may not admit the required density derivative; a sensitivity curve or a smoothed empirical law is then preferable.

Global contamination is measured by finite-sample maximum bias $b_n(m;y)={\sup }_{z:d_H(z,y)\le m}|T_n(z)-T_n(y)|$, where $d_H$ counts replacements. Under the convention of the first fraction that causes unbounded bias,This is the finite-sample replacement breakdown point. The largest replacement fraction still guaranteed bounded is one grid step smaller, so the conventions must be specified. One replacement sends a sample mean arbitrarily far, giving $1/n$, while the ordinary scalar sample median has first-breaking fraction $\lceil n/2\rceil /n$ with the usual midpoint convention for even samples. Its limiting value is $1/2$. Local influence and global breakdown answer different questions; a bounded influence function is not by itself a proof of a high breakdown point.

The expected check loss is finite because ${\rho }_{\tau }(u)\le |u|$. It is also Lipschitz in its location argument, with constant at most one. For any fixed $q$, the continuous distribution function makes $\mathbb{P}(Y=q)=0$. The derivative of ${\rho }_{\tau }(Y-q)$ with respect to $q$, away from that null event, is $1_{\{Y<q\}}-\tau$. Difference quotients are bounded by one, so the dominated convergence theorem givesThe continuous, strictly increasing distribution function has limits zero and one. Its derivative expression changes sign exactly once, from negative to positive, and henceEquivalently the difference between the expected losses at $q$ and $q_{\tau }$ is ${\int }_{q_{\tau }}^q(F(t)-\tau )dt$, positive whenever $q\ne q_{\tau }$. This proves population quantiles minimize check loss directly.

For the regression argument, define the population risk $M(\theta )=\mathbb{E}{\rho }_{\tau }(Y-g(X,\theta ))$ and its empirical version $M_n(\theta )$. Boundedness of $g$ and integrability of the error give integrability of $Y$ and of all these losses. Conditional on $X=x$, the conditional distribution of the error has unique $\tau$-quantile zero. Applying the preceding calculation conditionally shows that its expected loss is uniquely minimized when the fitted displacement $g(x,\theta )-g(x,{\theta }_0)$ equals zero. Thuswith strict inequality whenever that displacement is nonzero on an event of positive probability. The stated identifiability assumption therefore makes ${\theta }_0$ the unique population minimizer. This is conditional quantile identification.

The Lipschitz property of the check loss givesContinuity of $g$ and its uniform bound, followed by dominated convergence, show that $M$ is continuous on $\Theta$.

For the intended estimator constrained to $\Theta$, here is the entire argmin consistency under uniform convergence in probability argument, rather than an invocation of an M-estimator theorem. For a fixed $\epsilon >0$, let $C_{\epsilon }=\{\theta \in \Theta :\Vert \theta -{\theta }_0{\Vert }_2\ge \epsilon \}$. If this set is empty, there is nothing to prove. Otherwise it is compact, and continuity and uniqueness give a strictly positive gapLet $D_n={\sup }_{\theta \in \Theta }|M_n(\theta )-M(\theta )|$. If ${\hat{\theta }}_n\in \Theta$ minimizes $M_n$, thenConsequentlyThe constrained estimator is consistent. Existence of a constrained minimum follows from continuity of the sample criterion and compactness. The argument applies to any measurable choice of minimizer.

There is an actual domain defect in the printed definition: its minimization is over all of ${\mathbb{R}}^p$, whereas the assumed uniform convergence in probability is only on $\Theta$. That literal unconstrained consistency claim is false. The preceding conclusion needs minimization over $\Theta$, or another assumption that confines the selected minimizers to a set where the convergence and separation arguments apply. No change of question heading is needed to make this qualification explicit.

Here is a counterexample satisfying even identifiability over the whole line. Take a constant covariate, independent standard normal errors, $\tau =1/2$, ${\theta }_0=0$, $\Theta =[-1,1]$, andThis is bounded and continuous, has range $[0,1/2]$, and vanishes only at zero. The conditional error distribution function is continuous and strictly increasing with median zero. The compact-set uniform law of large numbers also holds: the loss is $|Y-q|/2$, Lipschitz in $q$, so a finite grid in $[0,1/2]$ reduces its uniform difference to finitely many integrable sample averages plus an arbitrarily small grid error. The weak law of large numbers then gives the asserted convergence.

Let $m_n$ be a sample median, and put $c_n=\min (1/2,\max (0,m_n))$. Absolute loss over the range $[0,1/2]$ is minimized at $c_n$. Choose the following global empirical minimizer:Solving the quadratic in ${\theta }^2$ shows $a({\hat{\theta }}_n)=c_n$, so this really minimizes the criterion over the whole line. For odd $n$, symmetry and continuity give $\mathbb{P}(m_n>0)=1/2$. Moreover $m_n\to 0$ in probability: for each fixed positive $\delta$, the proportion of observations below $\delta$ converges to a number greater than one half, and the analogous proportion below $-\delta$ converges to a number less than one half. Thereforealong odd $n$. The selected unconstrained estimator is not consistent. This illustrates why a global argmin may escape a compact convergence set.