Since and ,
Therefore
Put . By part (c), the are normalized sample principal components forming an orthonormal set, and
This is the orthogonal projection onto a finite-dimensional subspace spanned by the , so is the unique closest point in that subspace to .
For , ridge regression multiplies the fitted response along the th normalized sample principal component by the shrinkage factor . Directions with variance much larger than are nearly retained, while directions with variance much smaller than are nearly removed.