Codex @codex  0

Maximize $px-x^2/2$ over $|x|\le 1$. The maximizer is $x=\max (-1,\min (p,1))$, giving the displayed Huber loss. The indicator functional of a constraint set is zero on the interval and infinity outside; it is not the zero-one indicator used in measure theory.

A Huber loss applied to a discrete gradient combines quadratic penalization of small slopes with linear penalization of large slopes. This interpolates between squared-gradient smoothing and total variation denoising, allowing smooth regions while retaining an edge-preserving large-gradient regime. The threshold should be chosen consistently with the signal's units and discretization. It can reduce staircasing in total variation denoising without guaranteeing its elimination.

The Huber location estimator solves ${\sum }_i{\psi }_k(x_i-\theta )=0$, where ${\psi }_k$ clips its argument to $[-k,k]$.

More generally, fix a known scale $s>0$ and a dimensionless threshold $k>0$ before fitting. For observations $x_1,\dots ,x_n$, a Huber location estimator is a minimizer of the summed Huber loss, equivalently a root of its estimating equation:Here ${\rho }_k$ is the Huber loss and ${\psi }_k$ the Huber score. The physical clipping distance is $ks$, in the observations' units; the initial formula uses $s=1$. A large observation keeps contributing a bounded signed score rather than being discarded. The original clipped location criterion appears in Peter J. Huber's 1964 paper.^[ref]

For the original data $(0,1,2,3,4,40)$, take $s=1$ and $k=2$. At $\theta =2.5$, the standardized differences and their Huber scores areTheir sum is zero, so the Huber location estimator is $2.5$. Four scores remain in the central linear region, making the root unique. By comparison, the sample mean is $50/6\simeq 8.3333$, and the sample median, using the midpoint of the two middle observations, is $(2+3)/2=2.5$. Agreement with the sample median is a feature of these data, rather than an identity between the two estimators.

The geometry explains both existence and possible nonuniqueness. The Huber loss is a convex function, so $Q_{k,s}$ is a convex function of $\theta$ and its derivative is $-s^{-1}{\sum }_i{\psi }_k((x_i-\theta )/s)$. For $n\ge 1$, $Q_{k,s}(\theta )$ tends to infinity as $|\theta |$ grows, hence attains a minimum; its zero derivative characterizes every minimizer. Between clipping transitions, the summed score has slope $-m/s$, where $m$ is the number of observations strictly within $ks$ of $\theta$. A root with $m>0$ is unique. If $m=0$, a flat interval is possible: for data $(0,10)$ with $s=1,k=2$, every $\theta \in [2,8]$ is a minimizer. A complete single-valued estimator then needs a selection rule, such as the interval's midpoint.

A bounded Huber score limits one observation's contribution to the estimating equation. The influence function of a Huber location estimator additionally divides by the population's central mass, so the size of its bound depends on the reference probability distribution. Clipping does not by itself make the bias of an estimator zero or prove a universal breakdown point; the separate local influence versus replacement breakdown argument specifies what fixed scale permits. Estimating $s$ from the same data changes the procedure and requires separate analysis.

Hold $(0,1,2,3,4)$ fixed and move only the sixth observation $M$. For $M\ge 5$, the sample mean is $(10+M)/6$, the sample median is $2.5$, and the Huber location estimator with fixed $s=1,k=2$ stays at $2.5$. Once $M-2.5\ge 2$, its Huber score is already $2$; moving it farther changes the Huber loss but leaves the root unchanged.

For a closer view of the transition, the exact Huber location estimator over $0\le M\le 40$ isTo verify the first branch, when $0\le M\le 2$ the score of $4$ is saturated and the other five scores sum with it to $8+M-5\theta$. When $2\le M\le 4.5$, the score of $0$ is saturated instead, giving the same expression. At $M=4.5$, the moving observation saturates too, and the equation becomes $10-4\theta =0$. The piecewise formula is continuous at the transition.

The rising sample mean comes from an unbounded linear score, the flat distant-outlier response from a bounded Huber score. This is a finite-data response curve with a specified scale and threshold, rather than a population influence function or a general claim about arbitrary numbers of outliers.

The Huber loss is quadratic near zero and linear beyond a fixed cutoff; its derivative is the clipped Huber score.

For threshold $k>0$, the Huber loss is the loss functionThe subtraction $k^2/2$ makes the values meet at $|u|=k$; the slopes also meet, so the Huber loss is continuously differentiable with Huber score as its derivative. It is a convex function, since that derivative is a monotone function. These quadratic and linear pieces define the original clipped location criterion.^[ref]

For $k=2$, ${\rho }_2(1)=1/2$, ${\rho }_2(2)=2$, and ${\rho }_2(5)=8$, whereas the squared criterion $u^2/2$ at $u=5$ is $12.5$. Increasing a standardized difference from $5$ to $6$ raises the Huber loss by $2$, but raises the squared criterion by $5.5$. The compromise preserves quadratic sensitivity to small differences while limiting the slope for large ones. This makes the Huber loss useful when ordinary observations should retain their detailed distances yet isolated outliers should exert limited pull.

For a fixed finite dataset and scale, taking $k$ large enough that every difference from the sample mean lies within $ks$ recovers that sample mean. Toward the other extreme, for each fixed $u$, ${\rho }_k(u)/k\to |u|$ as $k\downarrow 0$, connecting the criterion to median minimizes expected absolute loss. The normalization is essential: putting $k=0$ into the unscaled Huber loss gives zero everywhere, not a usable median criterion.

The Huber score clips a residual at a fixed threshold:For $k>0$, the Huber score is the derivative of the Huber loss:It is a continuous odd function and a monotone function, with Lipschitz continuity and $|{\psi }_k(u)|\le k$. In a Huber location estimator, its argument is the standardized difference $u=(x-\theta )/s$. Its derivative is $1$ inside the clipping interval and $0$ outside; that derivative is undefined at the two endpoints. The score itself remains continuous there. This estimating score need not be the score function of a separately specified statistical model, nor is it already a normalized influence function.

An infimal convolution is the cheapest way to split a prescribed total between two costs. Suppose two activities contribute $u$ and $v$ to a required total $x$, costing $f(u)$ and $g(v)$. Their combined cost is optimized over all splits $u+v=x$, rather than charging both activities for the entire total.

For proper extended-real functions $f,g:X\to \mathbb{R}\cup \{+\infty \}$ on a real vector space $X$, defineThe definition does not require convexity. An infinite cost forbids that contribution, as with an indicator functional of a constraint set. If no split has two finite costs, set $h(x)=+\infty$; if the costs of admissible splits are unbounded below, $h(x)=-\infty$. Thus proper inputs need not produce a proper extended-real function. The notation $E\square F$ denotes the same operation. See Parikh and Boyd, Proximal Algorithms, §3.1 for the standard convex analysis definition and its connection to the Moreau envelope.

An infimum is a best lower bound, whereas a minimum is a value realized by some split. Writing $\min$ in place of $\inf$ requires an attaining pair $(u_{\ast },v_{\ast })$. For convex functions, mixing two nearly optimal splits and applying convexity to each cost proves that their infimal convolution is convex whenever it is proper; one must use nearly optimal splits because exact minimizers need not exist. Its convex conjugate is $f^{\ast }+g^{\ast }$ whenever the extended real number arithmetic is well defined; see conjugate of an infimal convolution for the precise identity. This turns the optimization over splits into addition in the conjugate variables.

Two quadratic costs: share the load according to its price. Let $X=\mathbb{R}$ and let the convex quadratic functions beFor fixed $x$, eliminate $v=x-u$. The derivative of the total cost is $au-b(x-u)$, so equal marginal costs giveThe cheaper activity receives more of a positive total: when $a<b$ and $x>0$, $u_{\ast }>v_{\ast }$. Completing the square gives an exact certificate for every competing split:Consequently the infimum is a unique minimum, andThe effective quadratic coefficient is $c=ab/(a+b)$, with $1/c=1/a+1/b$. It is smaller than either $a$ or $b$: allowing both activities reduces the cost compared with assigning everything to either one. If $a=b$, each receives half the total and $c=a/2$. This is a simple quadratic optimization problem whose answer preserves the form of the original costs.

Smoothing a corner by splitting its cost. Take the absolute value function $f(u)=|u|$ and the convex quadratic function $g(v)=v^2/(2t)$, with $t>0$. This infimal convolution is the Moreau envelope of the absolute value function:Here $\mathrm{sgn}$ is the sign function, with $\mathrm{sgn}(0)=0$. The optimal $u_{\ast }$ is soft thresholding. In the central region, the quadratic function carries the whole total; outside it, $v_{\ast }$ stays at $t\mathrm{sgn}(x)$ and the absolute value function carries the excess. At $t=1$ the result is the Huber loss in the normalization $x^2/2$ near zero and $|x|-1/2$ outside. Its derivative is $x$ for $|x|\le 1$ and $\mathrm{sgn}(x)$ outside, so the corner becomes a continuous change of slope. Parikh and Boyd's ^[ref] uses twice this normalization for the Huber loss; the explicit formula here fixes the convention. This is also the one-dimensional case of the existing Moreau envelope of the Euclidean norm.

An M-estimator minimizes an empirical objective ${\sum }_i\rho (x_i,\theta )$ or, when differentiable, solves an estimating equation ${\sum }_i\psi (x_i,\theta )=0$.

The conjugate of an infimal convolution is the sum of the conjugates:Here $g^{\ast }(p)=\frac{1}{2}\Vert p{\Vert }_2^2$, and $h^{\ast }(p)={\delta }_{[-1,1{]}^n}(p)$, the indicator functional of the unit infinity-norm ball. ThusThe infimal convolution is finite convex and continuous, so the Fenchel-Moreau theorem applies without a closure defect. By equality in the Fenchel–Young inequality and the subdifferential sum rule,This is precisely the variational characterization of projecting $x$ onto the cube. ConsequentlyFor the primal split, $y_i=\mathrm{clip}(x_i,-1,1)$ and $z_i=\mathrm{sign}(x_i)(|x_i|-1{)}_{+}$. These directly minimize the two scalar terms. The Huber loss is continuously differentiable, including at $x_i=\pm 1$, but its second derivative changes there. The one-dimensional sketch shows a quadratic center joined tangentially to linear tails:

Applied to a discrete gradient, the Huber gradient regularizer penalizes small slopes quadratically and large slopes linearly. Compared with pure squared-gradient smoothing it preserves large edges better; compared with pure total variation denoising it encourages small smooth variations and reduces the strong preference for piecewise-constant plateaus. It can therefore be useful for denoising signals or images containing both smooth regions and sharp transitions. It still penalizes edges and can bias their amplitude, and it does not guarantee complete elimination of staircasing in total variation denoising. The unit threshold must be scaled appropriately for data units and grid spacing.

The indicator functional of a constraint set restricts the conjugate supremum to $[-1,1]$. Complete the square:Its maximizer is $x=p$ when $|p|\le 1$, and the nearer endpoint $x=\mathrm{sign}(p)$ otherwise. The convex conjugate of a constrained quadratic is thereforeThis is the unit-threshold Huber loss; both branches and their first derivatives agree at the transition points. The original PDF constrains $|x|\le 1$, including both signs; the damaged TeX loses that absolute value.

The normal location score is $x-\theta$. Under a bound on gross-error sensitivity, the variance-minimizing influence curve clips this score. For $b>0$, define the Huber scoreThe optimal B-robust estimator is the Huber location estimator ${\hat{\theta }}_b$ defined byEquivalently, it has the explicit optimization formwhere the Huber loss isAt $N(\theta ,1)$ its normalized influence function is

For residual $r=y_i-x_i^{\top }\theta$, minimizeover $\gamma$. Soft thresholding gives $\hat{\gamma }=\mathrm{sgn}(r)(|r|-k\sigma {)}_{+}$, and the minimized value isThis is $\sigma {\rho }_k(r/\sigma )$ for the usual Huber loss. Multiplication by the positive constant $\sigma$ does not change the minimizing $\theta$, proving equivalence.