Vanishing-penalty ridge limit (${\lim }_{\lambda \downarrow 0}(X^{T}X+\lambda I{)}^{-1}{X}^{T}Y=(X^{T}X{)}^{+}{X}^{T}Y$)

In an eigenbasis of $G=X^{T}X$, ridge regression divides the corresponding component of $X^{T}Y$ by $\gamma +\lambda$. If $\gamma =0$, its eigenvector $v$ satisfies $Xv=0$, and hence $v^T{X}^{T}Y=0$ exactly. Only positive eigenvalues contribute to the limit, giving the Moore-Penrose inverse expression for the minimum-norm least-squares solution. No full-rank or sample-size assumption is needed.