Backpropagation

Backpropagation applies the chain rule from a neural network's output toward its input to compute the gradient of a loss with respect to every parameter.

Apply the chain rule to a recurrent network unrolled over its input sequence, adding contributions to parameters reused at multiple time steps. Products of state Jacobians convey effects from earlier states to later losses. This can compute exact gradients for the chosen finite unrolling, while physiological availability of the same history and error signals is a separate question.

A learning scheme replacing a transposed forward-weight matrix in the teaching signal with a separate fixed random matrix. Simulations show that adaptation of the forward weights can make this feedback useful on tested tasks. It need not equal the exact loss gradient, and successful simulations alone do not establish the mechanism in the brain.^[ref]

Exact backpropagation computes hidden teaching signals using the transpose of the current forward weight matrix. Explaining how a biological feedback pathway could know or maintain those weights is the weight transport problem. Useful approximate error correction need not satisfy exact reciprocal weight equality.^[ref]

For squared loss $E={\sum }_k(t_k-z_k{)}^2/2$, let $e_k=(t_k-z_k)g_k^{\prime }(x_k)$ and $e_j=g_j^{\prime }(x_j){\sum }_k{w}_{jk}{e}_k$. The chain rule gives $\partial E/\partial w_{jk}=-e_k{z}_j$ and $\partial E/\partial w_{ij}=-e_j{z}_i$. A gradient descent step of size $\eta$ uses $\Delta w_{jk}=\eta e_k{z}_j$ and $\Delta w_{ij}=\eta e_j{z}_i$, with every derivative evaluated at the same pre-update weights.

For hidden activations $h_j=\sigma (b_j+{\sum }_r{W}_{jr}{x}_r)$ and class probabilities $p_c={\mathrm{softmax}}_c(a+Vh)$, a one-hot label $t$ has log-likelihood $\ell ={\sum }_c{t}_c\log p_c$. Put ${\delta }_c=t_c-p_c$ and ${\Delta }_j=h_j(1-h_j){\sum }_c{V}_{cj}{\delta }_c$. ThenThese follow from the chain rule and give single-observation stochastic gradient descent updates for the negative log-likelihood.