For coordinate $j$, hold all other coefficients fixed and form the partial residual
$$
r_j=Y-\sum_{k\ne j}x_k\beta_k.
$$
The one-coordinate <coordinate descent> problem is
$$
\min_b\ \lVert r_j-x_jb\rVert^2+\lambda_1|b|+\lambda_2b^2.
$$
Its exact update is
$$
\beta_j\leftarrow
\frac{S_{\lambda_1}(x_j^Tr_j)}{\lVert x_j\rVert^2+\lambda_2},
\qquad
S_\lambda(u)=\operatorname{sgn}(u)(|u|-\lambda/2)_+.
$$
Cyclically update coordinates and their residuals until the objective or coefficients converge. Convexity makes every limit point a global minimizer, and $\lambda_2>0$ makes it the unique minimizer.

Solution

Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 218 / 5 / b / Solution

 Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2023/iii/paper-218/5/b/solution