Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 218 / 4 / b / Solution

The fitted classifier isand training minimizes the empirical categorical cross-entropy loss

The softmax non-identifiability already proves that the coefficient vector is not unique. For any $a\in {\mathbb{R}}^2$ and $r\in \mathbb{R}$, replace both rows of $V$ by $V_{k\cdot }+a^T$ and both output biases by $c_k+r$. Every logit then gains the same value $a^{T}h+r$, so all softmax probabilities, classifications, and losses remain unchanged. Permuting the two hidden units supplies another non-uniqueness.