= Quartic norm stabilization of Hebbian learning
{title2=$\tau\dot w=yx-\alpha y^2w|w|^2$}
With $y=w\cdot x$, the squared weight norm $s=|w|^2$ satisfies $\tau\dot s=2y^2(1-\alpha s^2)$. It moves monotonically toward $s_*=\alpha^{-1/2}$ without crossing it. Explicitly $(s-s_*)/(s+s_*)$ equals its initial value times $\exp[-(4\sqrt\alpha/\tau)\int_0^ty(u)^2\,du]$. Persistent excitation therefore drives $|w|$ to $\alpha^{-1/4}$; without excitation it can remain stationary. This nonlinear normalization avoids the unbounded amplification of unmodified <Hebbian learning>.
Back to article page
