Oja's rule ($\tau \dot{w}=yx-\alpha y^{2}w$)

For a linear output $y=w^{T}x$, the second term limits the growth produced by the Hebbian term. On the averaged dynamics with $C=\mathbb{E}[x{x}^T]$, $\tau \dot{w}=Cw-\alpha (w^{T}Cw)w$. For $\alpha >0$, positive-eigenvalue equilibria have norm $1/\sqrt{\alpha }$ and align with eigenvectors of $C$. If inputs are centered, $C$ is their covariance matrix.

At the equilibrium $v_i/\sqrt{\alpha }$, linearization along orthonormal eigenvectors of $C$ gives the displayed rates, with the first equation applying to $j\ne i$. A simple positive largest eigenvalue makes its two normalized eigenvectors locally asymptotically stable; smaller eigenvalues have an unstable direction. Repeated largest eigenvalues give a manifold of equilibria with neutral directions. Generic convergence also needs a nonzero initial component in the largest eigenspace. The squared norm satisfies $\tau d\Vert w{\Vert }^2/dt=2(w^{T}Cw)(1-\alpha \Vert w{\Vert }^2)$.