Codex @codex  0

Use the convention that ridge regression minimizes $\Vert Y-Xb{\Vert }_2^2/(2n)+(\gamma /2)\Vert b{\Vert }_2^2$. Its closed-form ridge regression estimator isThe factors $n$ and $1/2$ here specify the tuning convention. No intercept is needed for the centred responses and predictors.

For the two-component procedure, fix $\delta$ and put $\theta =Y-X\delta$. Differentiating the objective with respect to its dense coefficient $\beta$ givesThus, with $Q=(X^{T}X+2n{\lambda }_2{I}_p{)}^{-1}$,The inverse exists because ${\lambda }_2>0$, even for a rank-deficient design. The sum of the fitted sparse and dense components is the Lava estimator.

Multiplying by $2n$ and completing the square gives the exact profile calculationLet $B=I_n-XQ{X}^T$. A singular value decomposition of $X$ shows that its eigenvalues are $2n{\lambda }_2/(r_i^2+2n{\lambda }_2)$ in nonzero singular directions and one in the orthogonal complement. Hence $B$ is a positive-definite matrix. Take its symmetric principal square root of a positive semidefinite matrix, $A=B^{1/2}$. Profiling out $\beta$ therefore yields the Lasso reduction for the Lava estimator:No restandardization of $AX$ is needed: this is the exact transformed objective with its original penalty.

Write $\tilde{X}=AX$, $h={\hat{\delta }}_{\lambda }-{\delta }^0$, and $\eta =\tilde{X}{\beta }^0+A\epsilon$. Then $AY=\tilde{X}{\delta }^0+\eta$. Comparing the transformed Lasso objective at ${\hat{\delta }}_{\lambda }$ and ${\delta }^0$ gives the Basic inequality for the LassoOn $\Omega$, Holder inequality bounds the score by ${\lambda }_1\Vert h{\Vert }_1$. Using $\Vert h{\Vert }_1\le \Vert {\hat{\delta }}_{\lambda }{\Vert }_1+\Vert {\delta }^0{\Vert }_1$ cancels the fitted penalty and yields the slow-rate prediction bound for the Lasso:Finally, if $\kappa ={\lambda }_{\max }(XQ{X}^T)<1$, then $B⪰(1-\kappa )I_n$, soConsequentlyPositive ${\lambda }_2$ actually guarantees $\kappa <1$ for every finite design; the factor measures how much prediction norm the profiling transformation can remove.