= Solution
Let $z_{ij}$ be the input to output weight $w_j$ for datum $i$, including $z_{i0}=1$ for the bias. With identity output activation, $h_i=a_i=\sum_jw_jz_{ij}$. The <chain rule> gives
$$
\boxed{\frac{\partial E_i}{\partial w_j}=(h_i-y_i)z_{ij}}.
$$
The $z_{ij}$ are held fixed when differentiating an output-layer weight; they may be hidden-layer activations rather than raw descriptors. Hidden-weight derivatives require <backpropagation> through the preceding layers.
Back to article page
