= Leave-one-out residual identity for a linear smoother
{title2=$e_{(-i),i}=e_i/(1-H_{ii})$}
The familiar leave-one-out residual identity holds for ordinary or fixed-penalty <ridge regression>, but not for every arbitrary family of <linear smoothers>. Let $A=X^TX+\lambda\Omega$ be invertible, with a fixed penalty matrix $\Omega$ and fixed $\lambda$, and let $H=XA^{-1}X^T$. If deleting row $i$ leaves the normal matrix invertible, then $1-H_{ii}\ne0$ and
$$
Y_i-\widehat Y_{-i,i}=\frac{Y_i-\widehat Y_i}{1-H_{ii}}.
$$
The <Sherman–Morrison formula> applied to $A-x_ix_i^T$ gives the deleted coefficient estimate from the full one; multiplying by $x_i^T$ gives the identity. It permits <Leave-one-out cross-validation> without refitting all models.
The deleted fit must use the same penalty convention. Changing $\lambda$, selecting it again after deletion, or rescaling the loss while holding a normalized tuning parameter fixed changes the calculation. Merely writing the full fit as $HY$ does not specify its deletion rule and does not establish this identity.
Back to article page
