= Solution
Encode the two classes by the standard basis vectors of $\mathbb R^2$. With <rectified linear unit> $r(t)=\max(t,0)$ applied coordinatewise and <softmax function> $s_j(z)=e^{z_j}/\sum_ke^{z_k}$, the fitted <feedforward neural network> is
$$
\widehat p(x)=s\!\left(W_3r\!\left(W_2r(W_1x+b_1)+b_2\right)+b_3\right),
$$
where $W_1\in\mathbb R^{40\times40}$, $b_1\in\mathbb R^{40}$, $W_2\in\mathbb R^{20\times40}$, $b_2\in\mathbb R^{20}$, $W_3\in\mathbb R^{2\times20}$, and $b_3\in\mathbb R^2$. The number of trainable parameters is
$$
40\cdot40+40+20\cdot40+20+2\cdot20+2=2502.
$$
Back to article page
