= Nodewise-Lasso residual identity
{title2=$\widehat\tau_j^2=X_j^Tr_j/n$}
For <Nodewise Lasso> with objective $\|X_j-X_{-j}\gamma\|_2^2/(2n)+\lambda_j\|\gamma\|_1$, put $r_j=X_j-X_{-j}\widehat\gamma^{(j)}$. Its <Karush-Kuhn-Tucker conditions> give $X_{-j}^Tr_j/n=\lambda_jz_j$ with $z_j\in\partial\|\widehat\gamma^{(j)}\|_1$. Hence
$$
X_j^Tr_j/n=\|r_j\|_2^2/n+\lambda_j\|\widehat\gamma^{(j)}\|_1=\widehat\tau_j^2.
$$
For $\lambda_j>0$ this is positive whenever $X_j\ne0$; if it were zero both nonnegative terms would vanish, forcing $\widehat\gamma^{(j)}=0$ and $X_j=r_j=0$. Nonzero columns are therefore an essential hypothesis for division by this quantity.
Back to article page
