= Solution
Differentiating the <ridge regression> objective gives
$$
\nabla_\theta\bigl(\lVert Y-X\theta\rVert_2^2+\lambda\lVert\theta\rVert_2^2\bigr)
=2(X^TX+\lambda I_p)\theta-2X^TY.
$$
For every nonzero $z$,
$$
z^T(X^TX+\lambda I_p)z=\lVert Xz\rVert_2^2+\lambda\lVert z\rVert_2^2>0,
$$
so $X^TX+\lambda I_p$ is <positive-definite matrix>[positive definite] and hence invertible. The strictly convex objective therefore has the unique <closed-form ridge regression estimator>
$$
\boxed{\widehat\theta_\lambda=(X^TX+\lambda I_p)^{-1}X^TY.}
$$
Back to article page
