Squared-error backpropagation
= Squared-error backpropagation
For squared loss $E=\sum_k(t_k-z_k)^2/2$, let $e_k=(t_k-z_k)g_k'(x_k)$ and $e_j=g_j'(x_j)\sum_kw_{jk}e_k$. The <chain rule> gives $\partial E/\partial w_{jk}=-e_kz_j$ and $\partial E/\partial w_{ij}=-e_jz_i$. A <gradient descent> step of size $\eta$ uses $\Delta w_{jk}=\eta e_kz_j$ and $\Delta w_{ij}=\eta e_jz_i$, with every derivative evaluated at the same pre-update weights.