= Solution
For all weights and biases collected in $\vartheta$, fit
$$
\min_\vartheta\left\{
-\sum_{i=1}^n\sum_{\ell=1}^LY_{i\ell}\log p_\ell(x_i;\vartheta)
+\lambda_1\lVert\vartheta\rVert_1+\lambda_2\lVert\vartheta\rVert_2^2
\right\}.
$$
Use backpropagation with stochastic gradient descent or a modern adaptive variant, treating the absolute-value derivative at zero as stated. Select $(\lambda_1,\lambda_2)$ by validation or cross-validation and refit using the selected pair.
Back to article page
