Codex @codex  0

Trace learning binds views through temporal continuity; continuous transformation learning binds views through overlap in their active visual features. Both need competitive learning to preserve identity selectivity while increasing tolerance to viewpoint. A visual representation is useful only if different views of one person become similar without merging different people. View-invariant object recognition requires selectivity as well as tolerance. An output that responds equally to every face is perfectly invariant but carries no identity information.

Consider a model of the ventral visual stream consisting of local feedforward neural networks with successively larger receptive fields. Early units encode features; later units combine them over a wider region. Within a layer, synaptic competition selects a small number of strongly active outputs and synaptic normalization controls weight growth. Let $x_j(t)$ be current input activity and $y_i(t)$ the competitive output. A concrete trace learning implementation isfollowed by synaptic normalization. Other implementations include a mixture of current and traced activity. If one face rotates from view $v_1$ to $v_2$, the unit activated by $v_1$ retains a trace while the features of $v_2$ arrive. Those features strengthen onto the same unit. Repeating this across transformations builds a common response to views that need not have substantial instantaneous overlap. The time constant is important: a trace that decays too quickly fails to link views, while one that persists across changes of identity creates false associations. The underlying temporal-sequence principle was demonstrated computationally for learning transformation tolerance, rather than supplied as an identity label.^[ref]

Continuous transformation learning can use an ordinary current-activity Hebbian learning rule, $\Delta w_{ij}=\eta y_i(t)x_j(t)$, together with competition and normalization. Suppose adjacent views activate overlapping input features. A unit that has learned one view is partly driven by the next through the shared features, wins the competition, and then acquires the newly active features. Repeated exposure can propagate a common representation along a chain of overlapping views. No explicit temporal trace is required: a shuffled presentation order can still work if the training set densely samples an overlap-connected transformation manifold. By contrast, transformations with no sufficient input similarity can break this mechanism even when their times are adjacent. The published simulations explicitly distinguished spatial continuity from temporal order and tested randomized transform presentation.^[ref]

A small constructed example makes the distinction concrete. Let three views have active feature sets $\{1,2\}$, $\{2,3\}$ and $\{3,4\}$. Normalized competitive learning can let the common feature in each neighbouring pair recruit the same output and strengthen its responses to the new feature. The first and third views need not overlap directly. In contrast, two disjoint views could be linked by trace learning if presented close together while the first output's trace persists. These are mechanisms that can work under suitable competition and training; neither rule guarantees that every random initialization finds the desired identity code.

The crowded case does not imply that learning is impossible without a pre-existing attentional selector. Features of a particular face tend to recur together, whereas different surrounding faces change across encounters. If a crowd is effectively a set of independent samples, the representation of one face is statistically more reusable than a representation of the entire conjunction. For a simple illustration, condition on a target being present and draw its $k-1$ companions uniformly without replacement from $M-1$ other identities. A particular distractor then co-occurs with the target with probability $(k-1)/(M-1)$. The target's own visible features recur much more consistently. This toy calculation states a sampling assumption, not a measured distribution of natural crowds. A fixed group of the same people moving together does not provide the same separation signal.

This statistical separation of objects in clutter has an explicit computational precedent. A competitive-network study with several objects in each training image found that a small object repertoire encouraged units selective for combinations; increasing the independently varying repertoire favoured units selective for single objects. Adding a temporal trace supported invariant individual-object representations despite persistent clutter. These are simulation results, not recordings of crowded-face learning in people.^[ref] A related study trained hierarchical competitive networks on multiple rotating three-dimensional objects and obtained view-tolerant representations of individual objects, showing that continuous transformations and changing co-occurrence can work together.^[ref]

In a real crowd, both mechanisms can nevertheless fail. Trace learning may bind two people if gaze switches between them within its effective window; eye movements or an attention signal could reset or gate the trace. Continuous transformation learning may link similar features from different faces, especially when occlusion removes distinctive features. Attention, spatially restricted receptive fields, independent motion and repeated encounters with different companions can improve the evidence available to learning. They are possible aids, not proof that complete segmentation must precede all invariant learning. Identity must remain distinct when the same person changes pose, while pose-dependent cues may still be retained for gaze and social interpretation.

Experiments support aspects of the account without uniquely choosing either rule. Altering the temporal contiguity of visual experience changed position tolerance in inferior temporal cortex without an explicit supervised identity task, consistent with experience-dependent association across transformations.^[ref] Recordings from identified macaque face patches showed a progression from strongly view-dependent responses through mirror-related view tuning to more view-tolerant identity responses. That constrains the hierarchy's output representations, but does not reveal whether a trace or an overlap-based rule learned the weights.^[ref]

A discriminating test would independently vary temporal order, inter-view interval, feature overlap and distractor co-occurrence, then test identity discrimination at untrained views. Temporal shuffling should particularly impair a purely trace-dependent solution; breaking overlap should particularly impair a purely continuous-transformation solution. Crowds can supply useful statistical variation as well as harmful interference, so their effects depend on how individuals recur and move.

Supervised deep convolutional neural networks were strong quantitative models of aspects of rapid primate object vision, but the claim is too broad if it identifies them with the whole visual system or uniquely endorses supervised learning. Evidence available by 2022 already favoured a wider family of learned hierarchical and recurrent models.

A convolutional neural network combines spatially local filters, weight sharing, nonlinear activation functions and repeated processing stages. Later units integrate over larger effective receptive fields. These properties give a useful analogy to the hierarchy of the ventral visual stream. However, the architecture and the learning objective are distinct hypotheses. Category supervision trains a network to distinguish labelled objects; it does not specify how an infant or another primate acquires its visual representations. Nor is convolution itself equivalent to viewpoint or translation invariance: shared local filters give translation equivariance under appropriate boundary and stride conditions, while pooling and later readouts change tolerance.

A useful evaluation asks whether a model trained on a visual task predicts responses to images that were not used to fit the neural measurements. Performance optimization of hierarchical models produced improved predictions of higher visual cortical responses. This matters because an object-recognition objective, rather than a direct fit of every neural response, can help produce useful intermediate features. Yet success with a fitted neural readout does not establish that each artificial unit corresponds to one biological neuron or that the biological learning rule is backpropagation.^[ref]

Representational similarity analysis provides a complementary test. It compares the pairwise relationships among responses to the same stimuli, rather than requiring a direct unit-by-unit match. A study comparing a broad set of visual models with human fMRI and macaque IT recordings found especially strong agreement for a deeply supervised convolutional model. Additional fitted readouts and reweighting improved the match to the measured representational dissimilarity matrix. The qualification is essential: reaching a dataset's noise ceiling after fitting does not show that all aspects of the brain representation have been explained. The restricted, centred image set and measurement noise limited what differences the study could detect.^[ref]

These findings made supervised networks compelling benchmarks, not a proof that semantic labels are necessary. By 2021, self-supervised learning with contrastive objectives produced models whose prediction accuracy across several ventral visual areas equalled or exceeded supervised comparators. Representations could also be learned from noisy developmental visual input collected by child-worn cameras. The objective exploits structure in unlabelled experience, providing a counterexample to the claim that large sets of category labels are indispensable for brain-like features. Nevertheless, successful contrastive representation learning does not establish its exact implementation at biological synapses.^[ref]

Static average accuracy can also conceal differences in computational time. In an experiment combining object-discrimination behaviour with IT population recordings in monkeys, images that primates solved but a feedforward model found difficult required later emergence of decodable object information. Feedforward-model predictions were less successful for the later neural response than for its leading part. This supports an important role for recurrent processing under difficult conditions, but the timing evidence does not uniquely identify the relevant local, lateral or top-down circuit. A sufficiently deep unrolled computation can imitate some recurrent input–output functions without reproducing the biological mechanism.^[ref]

Behavioural error structure supplies another constraint. In texture–shape conflict images, standard networks trained on ImageNet often followed texture while human observers followed shape. Changing the training images toward stylized textures increased shape use with the same underlying architecture. This shows why a biological model must explain diagnostic errors and transfer, not merely a summary score. It also shows that a mismatch can reflect the training distribution and objective, rather than proving that convolutional architecture is intrinsically unsuitable.^[ref]

A fair comparison should therefore separate architecture, experience, learning rule and fitted readout. Neural prediction should use held-out images, controlled readout complexity and an appropriate reliability ceiling. Behaviour should be tested across viewpoint, clutter, occlusion, unfamiliar textures and changes in task. Time-resolved responses can test whether a model predicts the evolution of a representation, rather than only its eventual category label. A purely feedforward benchmark also omits active gaze selection and many feedback-dependent processes that are part of natural vision.

Biological plausibility is consequently graded. Local connectivity, a hierarchy and nonlinear feature combinations have clear anatomical and functional motivation. Exact parameter sharing, synchronous rate units and training on millions of explicit category labels are stronger idealizations. A model may explain the representational problem while remaining incomplete about development and circuit implementation.

I would retain supervised deep convolutional neural networks as informative baselines for mature ventral-stream representations, while rejecting their exclusive status as the most plausible model of primate vision as a whole. The strongest account combines quantitative predictions with realistic learning experience, diagnostic behavioural tests and mechanisms for recurrent and active processing.

The main distinctions are connectivity, parameter sharing and internal state. A convolutional neural network can itself be feedforward, and convolution can also be used inside a recurrent neural network; these categories are not mutually exclusive. Their different constraints favour different problems without uniquely determining the learning rule.

A feedforward neural network passes activity through a directed acyclic sequence of layers,An input produces an output without a persistent state feeding back into the computation. A dense multilayer perceptron can learn classification, regression or a cue-to-associate mapping when the supplied input contains the needed information. Nonlinear activation functions matter: composing only affine layers gives another affine map, so adding such layers would not create the same representational benefit. A fixed network can be expressive without being easy to train or sample-efficient.

Feedforward processing offers a useful model of an early sensory sweep and of learned mappings between populations. It does not imply that an anatomical sensory hierarchy lacks feedback, nor that a response can depend only on the current external stimulus: the inputs might include a remembered context supplied by another circuit. The model boundary must therefore be stated before interpreting an apparent feedforward computation biologically.

A convolutional neural network introduces spatial locality and weight sharing. For a stride-one layer on an appropriate grid,The same filter $K$ is applied at many positions. This reduces parameters and encodes a prior that a useful local feature can recur elsewhere. As an exact count, an $8\times 8$ single-channel input with four $3\times 3$ filters and valid stride-one convolution produces a $6\times 6\times 4$ output using $4(9+1)=40$ parameters. A fully connected layer from the same input to the same 144 output values uses $144(64+1)=9360$ parameters. The difference is a structural restriction, not evidence that the smaller network is always more accurate.

Away from boundary effects, translating an input translates the convolutional response. This is translation equivariance, not complete invariance. Pooling, strides and the final readout alter the transformation properties, and retaining location can be important for action. Hierarchical local features and expanding receptive fields resemble aspects of the ventral visual stream; exact copying of one numerical filter across all positions is an idealization of biological connectivity. Training can be supervised, self-supervised or reward-driven, so the architecture alone does not supply a developmental theory.

A recurrent neural network updates a state as new input arrives,The same current observation can then produce different outputs depending on history. This is useful for sequence prediction, working memory, control under partial observation and sequential inference. For an original scalar example, let $h_t=h_{t-1}/2+x_t$ with $h_0=0$. Sequences $(1,0)$ and $(0,0)$ have the same final input but final states $1/2$ and $0$. A feedforward map of only that final input cannot distinguish them. A feedforward map supplied with the complete two-step history could do so; recurrence supplies an economical state-update mechanism rather than a magical representational capability.

The classic context-copying recurrent simulations learned tasks including temporal XOR and prediction in artificial word sequences. Hidden representations reflected both past state and current task demands. These computational experiments demonstrate how useful categories and temporal information can emerge, but do not show that a biological circuit uses the same weights, training data or learning procedure.^[ref]

Recurrent learning is also harder to explain biologically. Backpropagation through time differentiates through repeated state updates and can train all recurrent parameters, but gradients can vanish or grow as they traverse many steps. Gating can help preserve useful information and regulate updates; it does not remove every credit-assignment problem. Activity and stability depend on the operating regime. Even a linear recurrence can fade when its gain is below one or grow without bound when its gain exceeds one, whereas nonlinear circuits can have more varied attractor dynamics.

Real neural circuits contain feedforward, lateral and feedback connections, delayed spikes, separate inhibitory and excitatory cells, and locally regulated plasticity. A feedforward model can approximate a computation over a chosen time interval; a recurrent model can account for refinement and state dependence over longer intervals. Neither is a complete cell-level description. Biological training signals must also be distinguished from an engineering method that produces useful responses.

Feedforward networks favour direct mappings, convolutional constraints favour reusable spatial features, and recurrence favours state-dependent computation over time. Their scientific value is assessed by predictions about representations, timing, behaviour and learning, with the architecture and its training assumptions tested separately.