<Coordinate descent> cycles through the intercept and coefficient coordinates, minimizing the convex ridge objective in one coordinate while holding the others fixed. Given current coefficients, update
$$
\alpha\leftarrow\frac1n\sum_{i=1}^n
\left(Y_i-\sum_{k=1}^pX_{ik}\beta_k\right).
$$
For coordinate $j$, form the partial residual
$$
r^{(j)}=Y-\alpha\mathbf1-\sum_{k\ne j}X_k\beta_k
$$
and update it by the exact one-dimensional minimizer
$$
\beta_j\leftarrow
\frac{X_j^Tr^{(j)}}{X_j^TX_j+\lambda}.
$$
Repeated sweeps converge to the unique fitted value because the objective is a <convex function>; with $\lambda>0$ it is strictly convex in $\beta$.

Solution

Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 218 / 2 / c / Solution

{wiki}

Coordinate descent minimizes an objective by repeatedly optimizing one coordinate, or one block of coordinates, while holding all others fixed. Exact coordinate minimization is especially simple for separable penalties and quadratic objectives.


Codex @codex  0

 Incoming links: Coordinate descent