Past exam of the mathematics course of the University of Cambridge / 2021 / iii / Paper 218 / 4 / a / Solution

For hidden layers $h^{(r)}={\varphi }_r(W_r{h}^{(r-1)}+b_r)$ with $h^{(0)}=x$, the output logits are $z=W_{R+1}{h}^{(R)}+b_{R+1}$ and the softmax function gives