 Oja's rule

For a linear output $y=w^{T}x$, the second term limits the growth produced by the Hebbian term. On the averaged dynamics with $C=\mathbb{E}[x{x}^T]$, $\tau \dot{w}=Cw-\alpha (w^{T}Cw)w$. For $\alpha >0$, positive-eigenvalue equilibria have norm $1/\sqrt{\alpha }$ and align with eigenvectors of $C$. If inputs are centered, $C$ is their covariance matrix.

Oja's rule is an unsupervised learning algorithm used in the field of neural networks and machine learning, particularly in the context of learning vector representations. It is a type of Hebbian learning rule, which is based on the principle that neurons that fire together, wire together. Oja's rule is specifically designed to allow a neural network to learn the principal components of the input data, effectively performing a form of principal component analysis (PCA).