Lasso regularization path ($\hat{\beta }(t)$)

The Lasso regularization path follows its coefficients as the penalty or constraint radius changes. Under the usual least-angle construction, the coefficient path is piecewise linear between knots, and variables can enter or leave the active set. Reading support at a selected cross-validation fraction requires checking which coefficients are actually zero at that fraction, rather than retaining only visibly large traces.

At a positive penalty level, let $E$ be the residual-score equality set, $s_j$ its signs and $N$ the nonzero regression coefficient support. With full column rank, the displayed strictly convex quadratic program selects the direction for a small decrease of the penalty. Existing nonzero regression coefficients have unrestricted tangent components; tied zero regression coefficients may move only with their residual signs. Stationarity gives $X_j^{T}Xv=s_j$ for $j\in N$ and for entering zero coordinates, and $s_j{X}_j^{T}Xv\ge 1$ for tied coordinates that remain zero. These are exactly the first-order Karush-Kuhn-Tucker conditions for the Lasso along the next segment. Normalize $Xv$ to obtain the corresponding least-angle direction. This resolves simultaneous entries or dropouts without adding a generic-position assumption.

// Target: mathematical-optimization.bigb

On a least-angle segment $\beta (\gamma )=\beta +\gamma d$, a nonzero active regression coefficient reaches zero at the displayed first positive root. Continue only as far as the first dropout, inactive-entry event or zero-correlation endpoint. At a dropout remove the vanishing regression coefficient from the active direction and recompute. This preserves agreement between nonzero regression coefficient signs and residual-score signs, which the Karush-Kuhn-Tucker conditions for the Lasso require. A removed regression coefficient may re-enter later; the active set need not grow monotonically.