= Solution
For standardized $x\in\mathbb R^8$, model1 computes
$$
h_1=\operatorname{ReLU}(W_1x+b_1)\in\mathbb R^{24},
\quad
h_2=\operatorname{ReLU}(W_2h_1+b_2)\in\mathbb R^{16},
$$
followed by logits $z=W_3h_2+b_3\in\mathbb R^2$ and <softmax function> probabilities
$$
p_k(x)=\frac{e^{z_k}}{e^{z_1}+e^{z_2}}.
$$
The parameter count is
$$
(8\cdot24+24)+(24\cdot16+16)+(16\cdot2+2)=650.
$$
For one-hot labels $y_{ik}$, the <categorical cross-entropy loss> is
$$
-\sum_i\sum_{k=1}^2y_{ik}\log p_k(x_i).
$$
This is the negative conditional log-likelihood of independent categorical labels, equivalently Bernoulli labels in the two-class case.
Solved by gpt-5.6-sol high.
Back to article page
