For all weights and biases collected in $\vartheta$, fit
$$
\min_\vartheta\left\{
-\sum_{i=1}^n\sum_{\ell=1}^LY_{i\ell}\log p_\ell(x_i;\vartheta)
+\lambda_1\lVert\vartheta\rVert_1+\lambda_2\lVert\vartheta\rVert_2^2
\right\}.
$$
Use backpropagation with stochastic gradient descent or a modern adaptive variant, treating the absolute-value derivative at zero as stated. Select $(\lambda_1,\lambda_2)$ by validation or cross-validation and refit using the selected pair.


Solution

Past exam of the mathematics course of the University of Cambridge / 2021 / iii / Paper 218 / 4 / d / Solution

 Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2021/iii/paper-218/4/d/solution