With column vectors and learned biases,
$$
h_1=\operatorname{ReLU}(W_1x+b_1),\quad
h_2=\operatorname{ReLU}(W_2h_1+b_2),\quad
h_3=\operatorname{ReLU}(W_3h_2+b_3),
$$
$$
z=W_4h_3+b_4,
\qquad \widehat p_l=\frac{e^{z_l}}{\sum_{r=1}^{26}e^{z_r}}.
$$
The dimensions are $16\to100\to75\to50\to26$. The number of parameters is
$$
\boxed{16\cdot100+100+100\cdot75+75+75\cdot50+50+50\cdot26+26.}
$$


Solution

Past exam of the mathematics course of the University of Cambridge / 2019 / iii / Paper 218 / 5 / a / Solution

 Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2019/iii/paper-218/5/a/solution