 Quartic norm stabilization of Hebbian learning

With $y=w\cdot x$, the squared weight norm $s=|w{|}^2$ satisfies $\tau \dot{s}=2y^2(1-\alpha s^2)$. It moves monotonically toward $s_{\ast }={\alpha }^{-1/2}$ without crossing it. Explicitly $(s-s_{\ast })/(s+s_{\ast })$ equals its initial value times $\exp [-(4\sqrt{\alpha }/\tau ){\int }_0^{t}y(u{)}^{2}du]$. Persistent excitation therefore drives $|w|$ to ${\alpha }^{-1/4}$; without excitation it can remain stationary. This nonlinear normalization avoids the unbounded amplification of unmodified Hebbian learning.