= Solution
The input vector $x$ contains the activities of presynaptic units or sensory features, and $w$ contains the corresponding synaptic weights. The linear output $y=w^Tx$ is their weighted sum; it may be a centered activity proxy rather than a literal nonnegative spike rate. The learning time scale is $\tau>0$, and take $\alpha>0$ for normalization. The term $yx$ is a <Hebbian learning> increment that strengthens weights correlated with the postsynaptic output. The term $-\alpha y^2w$ opposes unlimited growth.
For slow learning under stationary input statistics, average over inputs while holding the current weights fixed. Define $C=\langle xx^T\rangle$. The averaged <Oja's rule> is
$$
\tau\dot w=Cw-\alpha(w^TCw)w.
$$
Its squared norm obeys
$$
\boxed{\frac\tau2\frac{d\|w\|^2}{dt}=(w^TCw)(1-\alpha\|w\|^2).}
$$
Whenever the output second moment is positive, this drives the norm toward $1/\sqrt\alpha$, preventing the unbounded growth of an unconstrained Hebbian linear neuron. The direction becomes sensitive to dominant input correlations. Such a unit can learn a feature detector without supplied target labels, providing a simple mechanism for <unsupervised learning> and <principal component analysis>. This averaged analysis assumes input fluctuations are fast relative to learning; it is not a claim that every individual stochastic update follows the same deterministic trajectory.
Back to article page
