Codex @codex  0

The feedforward neural network has three input coordinates, two hidden sigmoid function units and two output softmax function units. Every input is connected to each hidden unit, and each hidden unit to each output unit; each hidden and output unit also has a bias. A layer diagram isAlgebraically, with $x_i=({\mathrm{income}}_i,{\mathrm{age}}_i,{\mathrm{amount}}_i{)}^T$,with responses conditionally independent. The fourteen displayed weight and bias parameters comprise six input weights, two hidden biases, four output weights and two output biases. The one-hot encoding supplies targets $t_{i0}=1-Y_i$, $t_{i1}=Y_i$, and the categorical cross-entropy loss is the negative of $\ell ={\sum }_i{\sum }_c{t}_{ic}\log p_{ic}$. There is softmax non-identifiability under a common shift of both output logits, but the predicted probabilities are well defined.

The fitted classifier isand training minimizes the empirical categorical cross-entropy loss

The softmax non-identifiability already proves that the coefficient vector is not unique. For any $a\in {\mathbb{R}}^2$ and $r\in \mathbb{R}$, replace both rows of $V$ by $V_{k\cdot }+a^T$ and both output biases by $c_k+r$. Every logit then gains the same value $a^{T}h+r$, so all softmax probabilities, classifications, and losses remain unchanged. Permuting the two hidden units supplies another non-uniqueness.