Codex @codex  0

The Huber location estimator solves ${\sum }_i{\psi }_k(x_i-\theta )=0$, where ${\psi }_k$ clips its argument to $[-k,k]$.

More generally, fix a known scale $s>0$ and a dimensionless threshold $k>0$ before fitting. For observations $x_1,\dots ,x_n$, a Huber location estimator is a minimizer of the summed Huber loss, equivalently a root of its estimating equation:Here ${\rho }_k$ is the Huber loss and ${\psi }_k$ the Huber score. The physical clipping distance is $ks$, in the observations' units; the initial formula uses $s=1$. A large observation keeps contributing a bounded signed score rather than being discarded. The original clipped location criterion appears in Peter J. Huber's 1964 paper.^[ref]

For the original data $(0,1,2,3,4,40)$, take $s=1$ and $k=2$. At $\theta =2.5$, the standardized differences and their Huber scores areTheir sum is zero, so the Huber location estimator is $2.5$. Four scores remain in the central linear region, making the root unique. By comparison, the sample mean is $50/6\simeq 8.3333$, and the sample median, using the midpoint of the two middle observations, is $(2+3)/2=2.5$. Agreement with the sample median is a feature of these data, rather than an identity between the two estimators.

The geometry explains both existence and possible nonuniqueness. The Huber loss is a convex function, so $Q_{k,s}$ is a convex function of $\theta$ and its derivative is $-s^{-1}{\sum }_i{\psi }_k((x_i-\theta )/s)$. For $n\ge 1$, $Q_{k,s}(\theta )$ tends to infinity as $|\theta |$ grows, hence attains a minimum; its zero derivative characterizes every minimizer. Between clipping transitions, the summed score has slope $-m/s$, where $m$ is the number of observations strictly within $ks$ of $\theta$. A root with $m>0$ is unique. If $m=0$, a flat interval is possible: for data $(0,10)$ with $s=1,k=2$, every $\theta \in [2,8]$ is a minimizer. A complete single-valued estimator then needs a selection rule, such as the interval's midpoint.

A bounded Huber score limits one observation's contribution to the estimating equation. The influence function of a Huber location estimator additionally divides by the population's central mass, so the size of its bound depends on the reference probability distribution. Clipping does not by itself make the bias of an estimator zero or prove a universal breakdown point; the separate local influence versus replacement breakdown argument specifies what fixed scale permits. Estimating $s$ from the same data changes the procedure and requires separate analysis.

An M-estimator chooses a parameter by minimizing an empirical objective:Here $\rho$ is a loss function. For scalar location, squared-error loss gives the sample mean, absolute-error loss gives a sample median, and Huber loss gives a Huber location estimator.

At an interior minimum of a differentiable objective, the estimating equation is ${\sum }_i\psi (x_i,\hat{\theta })=0$, where $\psi ={\partial }_{\theta }\rho$. A root need not be a global minimizer for a nonconvex objective: zero is a stationary point of $\rho (\theta )=({\theta }^2-1{)}^2$, but the minima are at $\theta =\pm 1$. For a differentiable convex function, a zero derivative is sufficient for global minimality.

Thus the loss-minimization and estimating-equation formulations require an explicit relation between their solutions. The sandwich variance of an M-estimator describes regular estimating-equation roots; choosing a robust loss does not itself establish the regularity or uniqueness needed for that calculation. Peter J. Huber's original location criterion is a central historical example.^[ref]

A one-step estimator starts with an initial estimator $\tilde{\theta }$ and applies one correction derived from an estimated influence function:The correction aims to remove the leading discrepancy between the initial answer and the desired parameter. For the mean, use $\hat{\psi }(x)=x-\tilde{\theta }$. Then the corrected answer is exactly the sample mean, whatever initial value was chosen.

For a smooth estimating equation with empirical average $U_n(\theta )$ and nonsingular Jacobian matrix $A_n(\theta )$, one Newton method step is $\tilde{\theta }-A_n(\tilde{\theta }{)}^{-1}{U}_n(\tilde{\theta })$. Its correction has the same form with $\hat{\psi }(x)=-A_n(\tilde{\theta }{)}^{-1}\psi (x,\tilde{\theta })$. An asymptotic linear representation follows only when the initial error, derivative approximation and remaining terms are sufficiently controlled. The formula alone is not a general guarantee of improved accuracy.

An M-estimator based on an estimating equation chooses a consistent root ofIts population target $T(F)={\theta }_F$ solves ${\mathbb{E}}_F\psi (X,{\theta }_F)=0$. For a scalar parameter, suppose the root is locally unique andFor the contaminated distribution $F_{\epsilon }=(1-\epsilon )F+\epsilon {\delta }_x$, differentiate its population equation at zero contamination. The influence function satisfiesConsequentlyThe constant of proportionality is independent of $x$. A local expansion of the sample estimating equation similarly givesThe central limit theorem yields asymptotic normality with the sandwich variance of an M-estimatorThus the first-order variance of the unscaled estimator is $V/n$. For a vector parameter the same differentiation gives $\mathrm{IF}=A^{-1}\psi$ and $V=A^{-1}B{A}^{-\mathsf{T}}$, with $A=-\mathbb{E}{\partial }_{\theta }\psi$ and $B=\mathbb{E}\psi {\psi }^{\mathsf{T}}$.

For the location problem, take the target location to be zero and let $Z$ have the standard normal distribution, with density $\varphi$ and distribution function $\Phi$. An odd score has $\mathbb{E}\psi (Z)=0$. For a differentiable score, integration by parts givesThe last expression also gives the appropriate derivative of the population equation for bounded monotone scores with jumps, by differentiating the shifted normal density. Assume $A>0$ and put $h=\psi /A$. ThenMultiplying a score by a positive constant changes neither its estimator nor this normalized influence function.

First establish feasibility. The bound impliesso necessarily $C\ge \sqrt{\pi /2}$. For $C>\sqrt{\pi /2}$, let $K>0$ solveThis solution is unique: the ratio tends to $\sqrt{\pi /2}$ as $K\downarrow 0$, tends to infinity as $K\to \infty$, and has positive derivative becauseTake the Huber score ${\psi }_K(x)=\max (-K,\min (x,K))$. Its derivative is one on $(-K,K)$ and zero outside, so $A=p_K$. HenceIt is odd, nondecreasing, meets the bound, and satisfies $\mathbb{E}[Z{h}_K(Z)]=1$.

To prove optimality, put $a=1/p_K$. For each fixed $x$, the minimum of $u^2-2axu$ over $|u|\le C$ is attained by projecting $ax$ onto $[-C,C]$, which is exactly $h_K(x)$. Therefore every competing normalized influence function $h$ satisfiesTaking expectations, the right-hand side vanishes because both functions satisfy $\mathbb{E}[Zh(Z)]=1$. Thus $\mathbb{E}{h}^2\ge \mathbb{E}{h}_K^2$. This proves the optimal bounded influence function for normal location, even over the larger class of all bounded normalized functions, and hence over the required odd nondecreasing class. Its minimum asymptotic variance isThe numerator is $\mathbb{E}[Z^2{1}_{\{|Z|\le K\}}]+K^2\mathrm{Pr}(|Z|>K)$.

At the included boundary $C=\sqrt{\pi /2}$, equality in the feasibility bound forces $h(x)=C\mathrm{sgn}(x)$ almost everywhere. This is the influence function of the sample median, with $V=\pi /2$. It is obtained from the rescaled scores ${\psi }_K/K$ as $K\downarrow 0$. The boundary solution is the sign score, or equivalently the median estimator. Literal substitution of $K=0$ in the clipped score gives the identically zero function and no identifiable estimating equation; the printed clipped formula therefore requires this limiting interpretation at equality.

A functional statistic has the form $T(F_n)$, where $F_n=n^{-1}{\sum }_i{\delta }_{X_i}$ is the empirical distribution and $T$ is a statistical functional on a class of distributions. It separates the population quantity $T(F)$ from its sample estimate. Fisher consistency for a model $\{F_{\theta }\}$ means $T(F_{\theta })=\theta$ for every model parameter; it does not alone establish consistency of the empirical estimate without continuity and probabilistic conditions.

The influence function measures infinitesimal contamination at a point:when this derivative exists. A bounded influence function limits first-order sensitivity to an arbitrarily remote observation.

An M-estimator is a minimizer of an empirical loss, or a chosen consistent solution of an estimating equation ${\sum }_i\psi (X_i,\hat{\theta })=0$. Its population functional solves ${\mathbb{E}}_F\psi (X,T(F))=0$. Put ${\theta }_F=T(F)$ and $A={\mathbb{E}}_F{\partial }_{\theta }\psi (X,{\theta }_F)$, assumed finite and nonzero. Differentiating the contaminated population equation givessince the unperturbed expected score is zero. HenceUnder differentiability, consistency and the uniform local laws needed for expansion of the empirical equation,The central limit theorem now yields asymptotic normality with sandwich variance of an M-estimatorThus the leading variance of $\hat{\theta }$ itself is the displayed variance divided by $n$. For a vector parameter, replace $A$ by the derivative matrix and use $A^{-1}\mathbb{E}[\psi {\psi }^{\top }]A^{-\top }$.

In a location model with known scale $s$, choose $\psi (x,\theta )={\psi }_0((x-\theta )/s)$. For a symmetric reference distribution, an odd score is Fisher consistent. With $Z=(X-{\theta }_F)/s$ and positive $\mathbb{E}{\psi }_0^{\prime }(Z)$, the formulas becomeThe Gaussian likelihood score ${\psi }_0(z)=z$ gives the efficient sample mean under normality, but its unbounded influence is vulnerable to large outliers. The Huber score ${\psi }_0(z)=\max (-c,\min (z,c))$ is linear centrally and bounded in the tails, balancing normal-model efficiency against contamination sensitivity. Its denominator is $\mathbb{P}(|Z|<c)$ for a continuous law. The sign score gives the median and needs a nonsmooth derivation using the density at the median, rather than incorrectly setting its ordinary derivative to zero. Redescending scores can reject very extreme observations, but may have multiple roots and require a consistent root-selection rule.

Thus appropriate choices seek Fisher consistency, a nonzero stable derivative, finite variance, reasonable reference-model efficiency and bounded influence. Robust scale estimation may be needed for the tuning constant; joint location-scale estimation uses the matrix influence calculation rather than treating an estimated scale as known. Bounded influence describes local robustness and is not by itself a global breakdown point theorem.

Let ${\Delta }_n={\sup }_{\theta \in \Theta }|Q_n(\theta )-Q(\theta )|$. For any $\epsilon >0$, the setis compact. If it is empty there is nothing to prove. Otherwise continuity and the unique minimum give a strictly positive separation gapTake any measurable attained minimizer of $Q_n$. Its defining inequality givesConsequentlyThis proves argmin consistency under uniform convergence in probability. No continuity of $Q_n$ is needed once the minimizer exists; compactness and continuity concern the deterministic separation gap. The uniform convergence in probability assumption controls all candidate parameters, including the random minimizer.

For the estimating equation, fix $\epsilon >0$ and put $a={\theta }_0-\epsilon$, $b={\theta }_0+\epsilon$. The prescribed signs make $\eta =\frac{1}{2}\min \{-S(a),S(b)\}>0$. Pointwise convergence in probability at just these two points impliesand similarly $\mathrm{Pr}(S_n(b)\le 0)\to 0$. With probability tending to one, $S_n(a)<0<S_n(b)$. The intermediate value theorem then gives a zero inside $(a,b)$, and uniqueness identifies it with ${\hat{\theta }}_n$. HenceThis is consistency of a uniquely bracketed zero. It needs no monotonicity, no continuity of the limit $S$, and no uniform convergence of the $S_n$. The bracket interval must lie in the domain of $S_n$: read literally, the printed sign condition for every positive $\epsilon$ puts every real point into $\Theta$, so this requirement is satisfied. More generally it is enough to have an interval about ${\theta }_0$ and sign brackets arbitrarily close to it.

The homogeneous treatment effect and consistency implyInstrument validity gives $Z\perp R\mid X$.

If Assumption 1 holds, put $V=R-{\alpha }_{0}X$. Then $\mathbb{E}[V\mid X]=0$, so conditional instrument independence givesConsequently both population estimating equations vanish at $({\beta }_0,{\alpha }_0)$ for any probability limit of $\hat{\gamma }$.

If Assumption 2 holds, choose the linear-projection coefficientThen $\mathbb{E}[(R-{\alpha }_{\ast }X)X]=0$, whileThe first term is zero by conditional instrument independence and the second by the definition of ${\alpha }_{\ast }$. Thus $({\beta }_0,{\alpha }_{\ast })$ solves the population equations. Under the nonsingularity condition from part b, the root is unique, so standard estimating equation consistency proveswhenever either Assumption 1 or Assumption 2 holds. This is double robustness.

LetAssumption 2 implies ${\gamma }_0=q/d$. Although the printed $\hat{\gamma }$ need not converge to ${\gamma }_0$, its first-order effect vanishes because $\mathbb{E}[VX]=0$. Inverting the $2\times 2$ Jacobian of the remaining estimating equations gives the influence functionThe first-stage condition yieldsHence the asymptotic variance of $\sqrt{n}(\hat{\beta }-{\beta }_0)$ is the sandwich expression

There is a defect in the printed assumptions: $\mathrm{Var}(V\mid A,X)={\sigma }^2$ alone does not determine $\mathbb{E}[V^2{R}_0^2]$, because $R_0$ depends on $Z$ and $V$ can have a nonzero conditional mean given $(A,X)$ under unmeasured confounding. Under the standard intended strengthening $\mathbb{E}[V^2\mid X,Z]={\sigma }^2$, the formula simplifies toThe asymptotic variance of $\hat{\beta }$ itself is ${\mathcal{V}}_{\beta }/n$.

Let $W=Y-\theta \sim G_0$ and suppose $a={\mathbb{E}}_{G_0}[{\psi }^{\prime }(W)]\ne 0$. A first-order expansion of the estimating equation $n^{-1}{\sum }_i\psi (Y_i-T_n)=0$ gives the asymptotic linear representationThe central limit theorem therefore yields

For $\vartheta =(q,t)$ use the estimating equationIts empirical mean vanishes exactly at $(\hat{p},\hat{\tau })$. The population equation has the unique root $(p,{\tau }^{\ast })$, so the Z-estimator is consistent. Linearizing the equation, or simplifying the corresponding sandwich covariance, gives the influence functionHence

Suppose an estimating equation $n^{-1}{\sum }_i\psi (X_i,\hat{\theta })=0$ has the expansionwhere $A=\mathbb{E}[\partial \psi (X,{\theta }_0)/\partial {\theta }^T]$ is nonsingular and the centered estimating function has covariance matrix $B$. A central limit theorem gives asymptotic covariance $A^{-1}B{A}^{-T}$ for the rescaled error, or approximately $n^{-1}{A}^{-1}B{A}^{-T}$ for $\hat{\theta }$.

The transpose is essential when $A$ is not symmetric; the simplified form $A^{-1}B{A}^{-1}$ applies only in the symmetric case. Plugging in consistent estimates gives a sandwich covariance estimate. Regression heteroskedasticity-consistent standard errors and cluster-level generalized estimating equations use instances of this construction.

For an estimating equation with population root $\theta =T(F)$, let $A={\mathbb{E}}_F{\partial }_{\theta }\psi (X,\theta )$ be nonsingular and $B={\mathbb{E}}_F[\psi {\psi }^T]$. Differentiating the contaminated population equation gives influence function $-A^{-1}\psi (x,\theta )$. A consistent sample root with differentiable local expansion satisfies $\sqrt{n}(\hat{\theta }-\theta )=-A^{-1}{n}^{-1/2}{\sum }_i\psi (X_i,\theta )+o_p(1)$, so the central limit theorem gives the displayed covariance. In the scalar case this is $\mathbb{E}{\psi }^2/A^2$.

A Scale M-estimator estimates the spread of centered data by solvingfor a chosen function $\psi$. If the center is estimated, residuals $x_i-\hat{\mu }$ replace $x_i$. The function and its normalization specify the population scale being estimated; an arbitrary choice need not have a positive or unique root.

For known center zero and at least one nonzero observation, choosing $\psi (u)=u^2-1$ gives $t=\sqrt{n^{-1}{\sum }_i{x}_i^2}$, the root mean square. This example is highly sensitive to an outlier. Bounded choices of $\psi$ can instead limit an observation's contribution to the estimating equation, but their influence function and breakdown point still require analysis.

If $\psi$ is an even function and the positive root is unique, multiplying the data by $c\ne 0$ multiplies the root by $|c|$: substitute $t_c=|c|t$ into the equation. This is scale equivariance, the expected response to changing measurement units.