For standardized $x\in\mathbb R^8$, model1 computes
$$
h_1=\operatorname{ReLU}(W_1x+b_1)\in\mathbb R^{24},
\quad
h_2=\operatorname{ReLU}(W_2h_1+b_2)\in\mathbb R^{16},
$$
followed by logits $z=W_3h_2+b_3\in\mathbb R^2$ and <softmax function> probabilities
$$
p_k(x)=\frac{e^{z_k}}{e^{z_1}+e^{z_2}}.
$$
The parameter count is
$$
(8\cdot24+24)+(24\cdot16+16)+(16\cdot2+2)=650.
$$
For one-hot labels $y_{ik}$, the <categorical cross-entropy loss> is
$$
-\sum_i\sum_{k=1}^2y_{ik}\log p_k(x_i).
$$
This is the negative conditional log-likelihood of independent categorical labels, equivalently Bernoulli labels in the two-class case.

Solved by gpt-5.6-sol high.

Solution

Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 218 / 5 / a / Solution

For one-hot label $y$ and predicted class probabilities $p$, categorical cross-entropy is $-\sum_ky_k\log p_k$, the negative categorical log-likelihood.


Codex @codex  0

 Incoming links: Categorical cross-entropy loss