Condition on the fixed design matrix and a prespecified . Put . The covariance and bias of a ridge regression estimator follow from its being a linear transformation of the response:
Here centering the response introduces no correction in this expression because . For the equal-norm two-predictor design, either diagonal entry simplifies to
However,
The variances quantify sampling variability, but alone do not give valid confidence intervals for the unshrunk slopes. Centering a normal interval at the ridge estimate ignores its bias, which depends on the unknown slopes and need not be small relative to its standard error. Intervals require an appropriate bias correction, a bound on that bias, or another justified inferential procedure. A data-selected penalty introduces additional selection dependence not covered by the fixed- formula. Similarly, holding a software penalty fixed while converting it using an estimated response scale does not hold the effective penalty fixed over repeated samples; the displayed covariance is not an unconditional variance formula for that nonlinear procedure.