Past exam of the mathematics course of the University of Cambridge / 2005 / ii / Paper 4 / 6E / c

Unmodified Hebbian learning reinforces correlated input and output but has no mechanism limiting synaptic strength. The second rule adds activity-dependent negative feedback and stabilizes the norm at an input-scale-independent value, provided the input keeps exciting the output. The third has a cubic weight penalty independent of the instantaneous output; its equilibrium norm instead depends on the strength of the input second moment.

For the third averaged rule, an equilibrium along eigenvector $i$ has radial linearized eigenvalue $-2{\lambda }_i/\tau$ and transverse eigenvalues $({\lambda }_j-{\lambda }_i)/\tau$. Thus a positive simple largest eigenvalue gives stable equilibria in its two opposite directions, whereas lower eigen-directions are unstable to larger components. With generic initial weights, it learns the dominant correlation direction, linking the rule to principal component analysis. These are idealized normalization and feature-extraction mechanisms, not claims that biological synapses implement these exact equations.