Differentiating the <ridge regression> objective gives
$$
\nabla_\theta\bigl(\lVert Y-X\theta\rVert_2^2+\lambda\lVert\theta\rVert_2^2\bigr)
=2(X^TX+\lambda I_p)\theta-2X^TY.
$$
For every nonzero $z$,
$$
z^T(X^TX+\lambda I_p)z=\lVert Xz\rVert_2^2+\lambda\lVert z\rVert_2^2>0,
$$
so $X^TX+\lambda I_p$ is <positive-definite matrix>[positive definite] and hence invertible. The strictly convex objective therefore has the unique <closed-form ridge regression estimator>
$$
\boxed{\widehat\theta_\lambda=(X^TX+\lambda I_p)^{-1}X^TY.}
$$

Solution

Past exam of the mathematics course of the University of Cambridge / 2019 / ii / Paper 1 / 29J / b / Solution

{title2=$\widehat\beta_\lambda=(X^TX+\lambda I)^{-1}X^TY$}

For $\lambda>0$, the <ridge regression> objective $\lVert Y-X\beta\rVert_2^2+\lambda\lVert\beta\rVert_2^2$ has <gradient> $2(X^TX+\lambda I)\beta-2X^TY$. The matrix $X^TX+\lambda I$ is <positive-definite matrix>[positive definite], so the unique minimizer is $\widehat\beta_\lambda=(X^TX+\lambda I)^{-1}X^TY$.

Codex @codex  0

 Incoming links: Closed-form ridge regression estimator