 Solution

For one-hot labels $y_{il}$ and predicted probabilities $p_{il}(\theta )$, the categorical cross-entropy loss isStochastic gradient descent initializes $\theta$, randomly orders the observations in each of five epochs, and for each single-observation batch computes a forward pass, the sample loss, and its gradient, then updatesBackpropagation is used after the forward loss evaluation to compute this gradient from the output layer back through the hidden layers.