Codex @codex  0

With $f=1_{\{\mathrm{font}=\mathrm{serif}\}}$ and $d=1_{\{\mathrm{display}=\mathrm{popup}\}}$, the model-matrix input isThe feedforward neural network has three inputs, a fully connected layer of two ReLU units, and a fully connected two-class softmax output. Algebraically,where $W\in {\mathbb{R}}^{2\times 3}$, $b\in {\mathbb{R}}^2$, $V\in {\mathbb{R}}^{2\times 2}$, and $c\in {\mathbb{R}}^2$. The coding is $0$ for no click and $1$ for a click. There areparameters.

Encode the two classes by the standard basis vectors of ${\mathbb{R}}^2$. With rectified linear unit $r(t)=\max (t,0)$ applied coordinatewise and softmax function $s_j(z)=e^{z_j}/{\sum }_k{e}^{z_k}$, the fitted feedforward neural network iswhere $W_1\in {\mathbb{R}}^{40\times 40}$, $b_1\in {\mathbb{R}}^{40}$, $W_2\in {\mathbb{R}}^{20\times 40}$, $b_2\in {\mathbb{R}}^{20}$, $W_3\in {\mathbb{R}}^{2\times 20}$, and $b_3\in {\mathbb{R}}^2$. The number of trainable parameters is

A signed spatial filter model of a simple cell assigns a weight $K_{ij}$ to each sampled location and takes its dot product with the visual contrast image $C_{ij}$. Positive weights favor bright features; negative weights favor dark features. The signed driveis a linear function of the image, while the rectified linear unit output $R$ is nonlinear. $D$ and $R$ below are arbitrary model units. To obtain a predicted firing rate one would need a calibrated gain, background response and possibly a different output function. The model does not generate a spike train; a linear-nonlinear-Poisson cascade model adds a stochastic event-generation assumption.

For the first diagram, every one of five rows has weightsA bright central column, with $C_{i3}=1$ and zero elsewhere, gives $D=5/5=1$. A bright middle row gives $D=-1/20-1/20+1/5-1/20-1/20=0$; a uniformly bright patch also gives $D=0$, since every row sums to zero. Move the bright column one site right: $D=5(-1/20)=-1/4$, so $R=0$. Make that shifted column dark instead: $D=1/4$ and $R=1/4$. Thus an OFF-like weight can increase output for darkness. A negative weight is a descriptive sensitivity, not evidence for a particular inhibitory synapse.

For a smooth example use the real, even Gabor filterThe Gaussian function localizes the weights; the cosine supplies alternating polarity. Choose ${\sigma }_x=1/2$, ${\sigma }_y=1$ and $q=\pi$ in arbitrary spatial units. The stripes' spatial frequency is $q/(2\pi )=1/2$ cycles per unit. Their axis makes angle $\theta$ with horizontal, while brightness changes across the perpendicular axis. $\phi$ is their spatial phase. Normalize the full-plane integral of $K{C}_{\theta ,\phi }$ by its value at $\theta =\pi /2$, $\phi =0$. The Gaussian integral and the product identity for cosines givewhereAt $\phi =0$, horizontal stripes give $R\approx 0.004159$, $60^{\circ }$ stripes give $R\approx 0.286741$, and vertical stripes give $R=1$. At the same vertical orientation, $\phi =\pi /2$ gives $D=R=0$, while reversing the polarity with $\phi =\pi$ gives $D=-1$ and $R=0$. The preferred axis persists while the response changes dramatically with position. Shifting the vertical stripes by a quarter of their wavelength changes phase by $\pi /2$ up to sign, without rotating them.

These are two different toy filters, not discretizations of the same fitted receptive field. The discrete filter exactly rejects a uniform contrast patch; this Gabor filter need not reject uniform contrast because its weight integral need not vanish. Neither toy includes stimulus history, recurrent interactions, saturation or a measured response baseline. Their value is a transparent prediction: orientation selectivity can emerge from spatial summation, and it does not by itself imply tolerance to phase.

A simple cell is a visual cortical neuron whose receptive field has spatially separable ON and OFF receptive-field subregions. Elongated neighboring subregions give a preferred orientation and a sensitivity to the location and polarity of visual contrast. “Simple” describes the response organization; it does not describe a simpler cell body or a particular shape of neuron.

Hubel and Wiesel's 1962 classification emphasized four observations: distinguishable ON and OFF receptive-field subregions, summation within each region, antagonism between regions, and qualitative prediction of responses to differently shaped stimuli from those regions' arrangement. A slit fitting an elongated center was more effective than a smaller spot; a larger patch could invade opposing flanks and lose effectiveness. An edge separating bright and dark regions could be effective when its boundary matched the boundary between subregions. These were observations about spike trains, rather than an assertion of exact linearity. ^[ref]

The following deliberately coarse signed spatial filter model of a simple cell makes the geometry easy to count. A centered bright bar contributes positively at five sites. Rotating it preserves its area but mixes positive and negative weights; shifting it preserves its orientation but changes which weights it samples. The model is useful because it predicts distinct effects of these three manipulations instead of treating any bright stimulus as equivalent.

A simple cell is not an ideal linear sensor. Its spike train is nonnegative in event count, and neural adaptation can change responsiveness. The signed spatial filter model of a simple cell supplies a useful approximation to spatial drive and an explicit output rule. Comparing predictions with unseen stimuli is more informative than assuming the rule from the name “simple”. In contrast, the existing complex cell models explore tolerance to phase and local displacement.