Codex @codex  0

The input vector $x$ contains the activities of presynaptic units or sensory features, and $w$ contains the corresponding synaptic weights. The linear output $y=w^{T}x$ is their weighted sum; it may be a centered activity proxy rather than a literal nonnegative spike rate. The learning time scale is $\tau >0$, and take $\alpha >0$ for normalization. The term $yx$ is a Hebbian learning increment that strengthens weights correlated with the postsynaptic output. The term $-\alpha y^{2}w$ opposes unlimited growth.

For slow learning under stationary input statistics, average over inputs while holding the current weights fixed. Define $C=⟨x{x}^T⟩$. The averaged Oja's rule isIts squared norm obeysWhenever the output second moment is positive, this drives the norm toward $1/\sqrt{\alpha }$, preventing the unbounded growth of an unconstrained Hebbian linear neuron. The direction becomes sensitive to dominant input correlations. Such a unit can learn a feature detector without supplied target labels, providing a simple mechanism for unsupervised learning and principal component analysis. This averaged analysis assumes input fluctuations are fast relative to learning; it is not a claim that every individual stochastic update follows the same deterministic trajectory.

A visual neuron's receptive field specifies the region of visual space and stimulus features whose changes affect its response, with the surrounding and temporal context held appropriately controlled. A linear receptive-field approximation is a signed spatial weighting kernel. A simple cell in primary visual cortex commonly has elongated alternating ON and OFF subregions and orientation selectivity. Brightening an ON region increases its drive; darkening an OFF region does likewise, while opposite contrasts oppose that drive in the linear approximation.

The original sketch below uses a Gaussian-windowed cosine to depict an elongated ON strip with OFF flanks. A bar aligned with these regions can produce strong net drive, whereas an orthogonal bar produces more cancellation. This is an illustrative signed filter, not a claim that every simple cell has exactly this analytical shape.

Different unsupervised learning models address different developmental aspects. Oja's rule can select a dominant direction of correlated input and provides normalization that a bare Hebbian update lacks. By itself, one linear principal component need not be localized, nor does it create a cortical map. Additional spatial constraints and competition are needed to explain a collection of spatially organized selective cells.

Competitive Hebbian learning among correlated ON and OFF inputs can separate those input types into adjacent subregions and generate orientation selectivity. Competition among inputs from the two eyes can similarly contribute to eye-specific preference and ocular dominance columns; spatial interactions between cortical units are needed for an organized map rather than merely one cell choosing one input pool. These models connect developmental organization to input correlations and normalization constraints. The ON/OFF competition model gives a concrete orientation-development example.

A different approach is sparse coding: learn features that reconstruct natural images while limiting simultaneous activity. This can yield localized, oriented, band-pass filters resembling simple-cell receptive fields, because it uses structure in natural-image statistics beyond the covariance alone. The original sparse-code model demonstrates this functional account. It does not by itself establish an exact developmental mechanism.

Correlated input, competition and normalization can produce selectivity; the objective and network constraints determine which aspect of cortical organization is explained. Anatomical constraints, spontaneous activity and experience must still be tested experimentally. A final receptive-field resemblance is weaker evidence than accurate predictions about its development and responses to altered input statistics.