Solution
= Solution
With column vectors and learned biases,
$$
h_1=\operatorname{ReLU}(W_1x+b_1),\quad
h_2=\operatorname{ReLU}(W_2h_1+b_2),\quad
h_3=\operatorname{ReLU}(W_3h_2+b_3),
$$
$$
z=W_4h_3+b_4,
\qquad \widehat p_l=\frac{e^{z_l}}{\sum_{r=1}^{26}e^{z_r}}.
$$
The dimensions are $16\to100\to75\to50\to26$. The number of parameters is
$$
\boxed{16\cdot100+100+100\cdot75+75+75\cdot50+50+50\cdot26+26.}
$$