= Covariance and bias of a ridge regression estimator
{title2=$\operatorname{Cov}(\widehat\beta_\lambda)$}
For a fixed centered <design matrix> $X$, independent errors of <variance> $\sigma^2$, and objective $\lVert Y-X\beta\rVert^2+a\lVert\beta\rVert^2$, put $G=X^TX$ and $M=G+aI$. The <ridge regression> estimator has
$$
\operatorname{Cov}(\widehat\beta)=\sigma^2M^{-1}GM^{-1},\qquad \mathbb E\widehat\beta-\beta=-aM^{-1}\beta.
$$
Thus its standard error alone does not justify centering a <confidence interval> for $\beta$ at the biased estimator. A penalty chosen from the responses also changes its <sampling distribution>.
Back to article page
