Unbounded directional risk of fixed ridge shrinkage
= Unbounded directional risk of fixed ridge shrinkage
{title2=$\sup_{\beta^0,\|x\|_2=1}(R_R(x)-R_{OLS}(x))=\infty$}
For fixed positive <ridge regression> penalty, choose a unit <eigenvector> $v$ of $X^{\mathsf T}X$ with eigenvalue $d>0$, and signal $\beta^0=Bv$. The directional ridge risk is $(\lambda^2B^2+\sigma^2d)/(d+\lambda)^2$, whereas <ordinary least squares> has risk $\sigma^2/d$. Letting $|B|$ grow makes the ridge squared bias arbitrarily large.