= Basic inequality for a penalized least-squares estimator
{title2=$\|h\|_2^2/n\leq2\langle z,h\rangle/n+\lambda J(\theta^*)-\lambda J(\widehat\theta)$}
For the identity design and $y=\theta^*+z$, comparing the minimizing objective at $\widehat\theta$ with its value at $\theta^*$ and expanding the squared <Euclidean norm> proves the displayed inequality, with $h=\widehat\theta-\theta^*$. This deterministic inequality does not assume any <probability distribution> for the noise. A bound on the noise inner product turns it into a statistical error estimate.
Back to article page
