= Solution
For residual $r=y_i-x_i^\top\theta$, minimize
$$
\frac{(r-\gamma)^2}{2\sigma}+k|\gamma|
$$
over $\gamma$. <Soft thresholding> gives $\widehat\gamma=\operatorname{sgn}(r)(|r|-k\sigma)_+$, and the minimized value is
$$
\begin{cases}
r^2/(2\sigma),&|r|\leq k\sigma,\\
k|r|-k^2\sigma/2,&|r|>k\sigma.
\end{cases}
$$
This is $\sigma\rho_k(r/\sigma)$ for the usual <Huber loss>. Multiplication by the positive constant $\sigma$ does not change the minimizing $\theta$, proving equivalence.
Back to article page
