= Solution
Set $s=|w|^2$. The exact <Hebbian learning> rule gives
$$
\tau\dot s=2w\cdot(yx)=2y^2\ge0.
$$
Thus the <norm> never decreases, and increases whenever the output is nonzero. Because learning is slow relative to input fluctuations, its averaged equation is $\tau\dot w=Cw$, where $C=\langle xx^T\rangle$ is the input second-moment <matrix>. It is a <covariance matrix> only if the inputs have zero mean. Diagonalizing the <positive semidefinite matrix> $C$ gives components $w_j(t)=w_j(0)e^{\lambda_jt/\tau}$. Hence \b[weights grow without bound whenever the initial <vector> has a component in an excited positive-eigenvalue direction]. A <vector> entirely in $\ker C$, or identically zero input, is an important stationary exception.
Back to article page
