Basic inequality for the Lasso
ID: basic-inequality-for-the-lasso
Suppose and minimizes the Lasso objective . With , comparing its value with the value at givesIndeed, expand and cancel . This deterministic inequality needs neither a normal distribution nor a full-rank design matrix.[ref]
Statistical Theory for High-Dimensional Models by Sara van de Geer. Section 1.3, basic inequality proof
The noise term is at most by Holder inequality. Controlling that term probabilistically leads to a Lasso cone condition; a Compatibility condition for the Lasso then converts coefficient error into prediction error. The stronger prediction inequality from Lasso stationarity uses subgradient optimality and has a different leading constant.
New to topics? Read the docs here!