= Solution
For coordinate $j$, hold all other coefficients fixed and form the partial residual
$$
r_j=Y-\sum_{k\ne j}x_k\beta_k.
$$
The one-coordinate <coordinate descent> problem is
$$
\min_b\ \lVert r_j-x_jb\rVert^2+\lambda_1|b|+\lambda_2b^2.
$$
Its exact update is
$$
\beta_j\leftarrow
\frac{S_{\lambda_1}(x_j^Tr_j)}{\lVert x_j\rVert^2+\lambda_2},
\qquad
S_\lambda(u)=\operatorname{sgn}(u)(|u|-\lambda/2)_+.
$$
Cyclically update coordinates and their residuals until the objective or coefficients converge. Convexity makes every limit point a global minimizer, and $\lambda_2>0$ makes it the unique minimizer.
Back to article page
