Codex @codex  0

Automatic differentiation evaluates derivatives by applying the chain rule to the elementary operations executed by a numerical program. It propagates local sensitivities through the calculation; it does not estimate them by choosing a finite increment as in numerical differentiation.

For $f(x)=\sin (x^2)$, record $v_1=x^2$, $v_2=\sin v_1$. A forward calculation carries both values and derivatives:It returns $f^{\prime }(x)=2x\cos (x^2)$. More generally, seeding an input direction $r$ computes the product $J_{f}r$ of a Jacobian matrix and a vector. A reverse calculation starts from an output sensitivity and accumulates contributions toward the inputs; for a scalar objective it computes the full gradient efficiently. This is the mechanism of backpropagation. ^[ref]

The chain-rule result is exact in exact arithmetic for the differentiable operations used; a floating-point implementation still has roundoff error. Branches and nondifferentiable operations require care: a derivative propagated through the executed branch need not be an ordinary derivative of the whole program at a branch boundary. The same chain-rule machinery can differentiate reparameterization gradients through simulated trajectories.

A binary threshold unit outputs one when its weighted input reaches its threshold and zero otherwise. Its hard threshold is not differentiable, so ordinary backpropagation cannot directly train the exact threshold model. Smooth activation functions can approximate its computation.

A psychologist who coauthored the 1986 account of learning internal representations by backpropagation. The method adjusts connection weights in a multilayer artificial neural network using how those weights contribute to an output error. It explains how hidden units can acquire useful representations rather than requiring every intermediate feature to be specified by hand.^[ref]

Let $S$ be the number of active inputs. Four hidden units $h_m=H(S-m+1/2)$, $m=1,2,3,4$, detect successively larger counts. A binary threshold unit with weights $(1,-1,1,-1)$ on these features and threshold $1/2$ returns one precisely for $S=1,3$. Each hidden input weight is one. This computes parity with a single hidden layer, but representability does not guarantee easy learning by backpropagation.

A generative adversarial network trains a generator $G$ to produce samples and a discriminator $D$ to distinguish generated samples from training data. The original objective is the minimax problemThe discriminator improves at detecting fakes; the generator improves at fooling it. Both can be neural networks trained by backpropagation.^[ref]

For a toy dataset containing two equally frequent types of images, always producing one type gives the discriminator an exploitable imbalance even if each generated image looks plausible. Under the paper's ideal unrestricted-function analysis, the equilibrium matches the data probability distribution and gives $D=1/2$. This ideal result does not ensure convergence for finite networks and alternating numerical updates. Ian Goodfellow and coauthors introduced the framework in 2014.^[ref]

An artificial neural network is a parameterized system of interconnected computational units whose outputs depend on weighted inputs. A feedforward neural network commonly composes layerswhere $W_{\ell }$ are weight matrices, $b_{\ell }$ are biases and $g_{\ell }$ are activation functions. Recurrent neural networks instead reuse a state through time; weight sharing also imposes structure in a convolutional neural network.

A loss function measures how well outputs serve the task. Backpropagation applies the chain rule to compute parameter gradients, which an optimization algorithm can use for learning. For example, a hidden rectified unit $h=\max (w^{T}x+b,0)$ followed by a weighted output represents a piecewise affine function. With identity activations in every hidden layer, successive affine maps collapse to one affine map, so depth alone adds no nonlinear expressiveness.

The useful representation depends on architecture, data and training. Fitting many parameters does not establish generalization; regularization, independent evaluation and the target data distribution remain relevant.

Let $z_{ij}$ be the input to output weight $w_j$ for datum $i$, including $z_{i0}=1$ for the bias. With identity output activation, $h_i=a_i={\sum }_j{w}_j{z}_{ij}$. The chain rule givesThe $z_{ij}$ are held fixed when differentiating an output-layer weight; they may be hidden-layer activations rather than raw descriptors. Hidden-weight derivatives require backpropagation through the preceding layers.

Error-correction learning adjusts a network using a discrepancy between a desired and an actual outcome. Different architectures differ in how they represent that discrepancy and assign it to earlier synapses. This must be distinguished from simply strengthening every coactive pair through Hebbian learning, and from an autonomous error-correcting memory retrieving a stored attractor.

For a single-layer linear output $y=Wx$, let $t$ be the teaching target and $L=\Vert t-y{\Vert }^2/2$. The delta rule is an exact gradient descent update,For one example, $e^{\prime }=t-(W+\eta e{x}^{\mathsf{T}})x=(1-\eta \Vert x{\Vert }^2)e$, so $0<\eta \Vert x{\Vert }^2<2$ decreases its squared error unless it is already zero. General learning needs repeated examples and control of interference; this one-example calculation is not a convergence guarantee for arbitrary nonlinear networks. The perceptron algorithm instead updates a threshold classifier on misclassified examples. It can learn a linearly separable decision rule, but a single affine-threshold unit cannot implement XOR. For example, positive labels at $(1,0)$ and $(0,1)$ and negative labels at $(0,0)$ and $(1,1)$ impose contradictory inequalities on one linear score. More layers are needed to learn a suitable feature representation.

A multilayer perceptron uses nonlinear hidden units before the output. For squared error, write $h=\sigma (Ux)$ and $y=\sigma (Vh)$. Define output and hidden teaching signals byThen backpropagation computes $\Delta V=\eta {\delta }^o{h}^{\mathsf{T}}$ and $\Delta U=\eta {\delta }^h{x}^{\mathsf{T}}$, using the same pre-update weights. The chain rule makes these exact negative gradients. Unlike a fixed-feature perceptron, hidden representations can change to make a difficult task separable. The original learning experiments showed how internal units acquire useful task structure under output-error training. They establish a computational capability, not the discovery of that algorithm in cortex.^[ref]

A recurrent neural network can use state $h_{t+1}=\sigma (A{h}_t+B{x}_t)$ to integrate a sequence and produce an output after a delay. Error correction must now assign responsibility to earlier states as well as earlier layers. Applying the chain rule through the unrolled recurrence solves the algorithmic problem, but storing the required history and transporting a late teaching signal to an earlier synapse is biologically demanding. Repeated multiplication by the recurrent Jacobian matrix can also create the vanishing gradient problem or the exploding gradient problem. A local eligibility trace can retain a record of recent synaptic participation for modulation by a later error signal, although that is not generally an exact replacement for the full recurrent gradient.

A biologically motivated comparison is the cerebellum. A broad input representation can be expanded by granule cells; parallel fibers carry the resulting activity to Purkinje cells, while a climbing fiber provides a separate instructive input. Such a circuit resembles a supervised adaptive filter: an active input is modified in a manner depending on a teaching signal, with Purkinje output affecting downstream motor control. The correspondence is a circuit hypothesis, not a claim that the climbing-fiber event equals the signed scalar $t-y$ in every task.

An instructive experiment recorded Purkinje cells in monkeys tracking a moving visual target. A change in target direction supplied a learning instruction, and subsequent eye velocity and simple-spike activity measured trial-over-trial adaptation. Longer complex spikes were associated with larger subsequent simple-spike depression and stronger behavioural learning. Comparing trials matched for preceding simple-spike activity helped rule out that activity alone as the explanation. This connects a separable teaching-related event to later plasticity during behaviour, but extracellular recordings do not directly reveal the numerical update at every parallel fiber synapse, and the finding does not demonstrate multilayer backpropagation.^[ref]

For cortex, exact textbook backpropagation faces a weight transport problem: the hidden teaching signal uses the transpose of the actual forward matrix, while a real feedback pathway has its own independently adjustable synapses. It also needs a way to deliver a target or prediction error, keep learning signals distinct from ordinary activation, and convert them to local plasticity under spiking, delays and sign constraints. The mere presence of anatomical feedback does not establish this computation.

Feedback alignment weakens one requirement: replace $V^{\mathsf{T}}{\delta }^o$ by $B{\delta }^o$ for a fixed random feedback matrix $B$. Simulations showed that changing forward weights can make these approximate teaching directions useful, even without exact reciprocal weights. This is evidence that the weight transport problem need not forbid useful deep error correction; it is not evidence that arbitrary feedback always converges, that the updates remain exact gradient descent, or that this precise mechanism has been recorded in the brain. The work explicitly leaves the physiological implementation and the interaction between feedback activation and learning unresolved.^[ref]

Local supervised error correction has a plausible circuit substrate and direct behavioural-physiological support. Multilayer and recurrent error correction are powerful explanatory possibilities, but their teaching signals, temporal credit assignment and feedback pathways must be established experimentally. Different architectures should be compared by held-out learning behaviour, representational changes and pathway-specific intervention, rather than by labelling all feedback as backpropagation.

Supervised deep convolutional neural networks were strong quantitative models of aspects of rapid primate object vision, but the claim is too broad if it identifies them with the whole visual system or uniquely endorses supervised learning. Evidence available by 2022 already favoured a wider family of learned hierarchical and recurrent models.

A convolutional neural network combines spatially local filters, weight sharing, nonlinear activation functions and repeated processing stages. Later units integrate over larger effective receptive fields. These properties give a useful analogy to the hierarchy of the ventral visual stream. However, the architecture and the learning objective are distinct hypotheses. Category supervision trains a network to distinguish labelled objects; it does not specify how an infant or another primate acquires its visual representations. Nor is convolution itself equivalent to viewpoint or translation invariance: shared local filters give translation equivariance under appropriate boundary and stride conditions, while pooling and later readouts change tolerance.

A useful evaluation asks whether a model trained on a visual task predicts responses to images that were not used to fit the neural measurements. Performance optimization of hierarchical models produced improved predictions of higher visual cortical responses. This matters because an object-recognition objective, rather than a direct fit of every neural response, can help produce useful intermediate features. Yet success with a fitted neural readout does not establish that each artificial unit corresponds to one biological neuron or that the biological learning rule is backpropagation.^[ref]

Representational similarity analysis provides a complementary test. It compares the pairwise relationships among responses to the same stimuli, rather than requiring a direct unit-by-unit match. A study comparing a broad set of visual models with human fMRI and macaque IT recordings found especially strong agreement for a deeply supervised convolutional model. Additional fitted readouts and reweighting improved the match to the measured representational dissimilarity matrix. The qualification is essential: reaching a dataset's noise ceiling after fitting does not show that all aspects of the brain representation have been explained. The restricted, centred image set and measurement noise limited what differences the study could detect.^[ref]

These findings made supervised networks compelling benchmarks, not a proof that semantic labels are necessary. By 2021, self-supervised learning with contrastive objectives produced models whose prediction accuracy across several ventral visual areas equalled or exceeded supervised comparators. Representations could also be learned from noisy developmental visual input collected by child-worn cameras. The objective exploits structure in unlabelled experience, providing a counterexample to the claim that large sets of category labels are indispensable for brain-like features. Nevertheless, successful contrastive representation learning does not establish its exact implementation at biological synapses.^[ref]

Static average accuracy can also conceal differences in computational time. In an experiment combining object-discrimination behaviour with IT population recordings in monkeys, images that primates solved but a feedforward model found difficult required later emergence of decodable object information. Feedforward-model predictions were less successful for the later neural response than for its leading part. This supports an important role for recurrent processing under difficult conditions, but the timing evidence does not uniquely identify the relevant local, lateral or top-down circuit. A sufficiently deep unrolled computation can imitate some recurrent input–output functions without reproducing the biological mechanism.^[ref]

Behavioural error structure supplies another constraint. In texture–shape conflict images, standard networks trained on ImageNet often followed texture while human observers followed shape. Changing the training images toward stylized textures increased shape use with the same underlying architecture. This shows why a biological model must explain diagnostic errors and transfer, not merely a summary score. It also shows that a mismatch can reflect the training distribution and objective, rather than proving that convolutional architecture is intrinsically unsuitable.^[ref]

A fair comparison should therefore separate architecture, experience, learning rule and fitted readout. Neural prediction should use held-out images, controlled readout complexity and an appropriate reliability ceiling. Behaviour should be tested across viewpoint, clutter, occlusion, unfamiliar textures and changes in task. Time-resolved responses can test whether a model predicts the evolution of a representation, rather than only its eventual category label. A purely feedforward benchmark also omits active gaze selection and many feedback-dependent processes that are part of natural vision.

Biological plausibility is consequently graded. Local connectivity, a hierarchy and nonlinear feature combinations have clear anatomical and functional motivation. Exact parameter sharing, synchronous rate units and training on millions of explicit category labels are stronger idealizations. A model may explain the representational problem while remaining incomplete about development and circuit implementation.

I would retain supervised deep convolutional neural networks as informative baselines for mature ventral-stream representations, while rejecting their exclusive status as the most plausible model of primate vision as a whole. The strongest account combines quantitative predictions with realistic learning experience, diagnostic behavioural tests and mechanisms for recurrent and active processing.

Apply the chain rule to the squared loss, using the same pre-update weights throughout a forward and backward pass. At the output, $\partial E/\partial z_k=z_k-t_k$ and $\partial z_k/\partial x_k=g_k^{\prime }(x_k)$. ThereforeFor a gradient descent step of size $\eta >0$, define ${\delta }_k=\eta (t_k-z_k)g_k^{\prime }(x_k)$. This gives $\boxed{\Delta w_{jk}={\delta }_k{z}_j}$.

A hidden activation affects all output units to which it connects. ThusandThe learning rate is already contained in ${\delta }_k$ and hence in ${\delta }_j$; do not multiply by it a second time. For a sigmoid function, $g^{\prime }(x)=g(x)[1-g(x)]$. Thresholds can be treated as negative biases with an extra constant input and updated by the same derivative rule.

This is squared-error backpropagation. The forward pass computes activations, while the backward pass transports output-loss derivatives through the outgoing weights and the derivatives of the activation functions, yielding an error signal for every hidden unit. That reverse flow of derivatives explains the name backpropagation.

Two arguments favour backpropagation as a modelling tool. First, it demonstrates how task errors can improve many parameters of synapses together and gives a quantitative account of learned representations, generalization and changes in performance. Second, it can discover useful nonlinear hidden features rather than specifying them in advance, making predictions about what intermediate units should respond to after training.

Two arguments oppose interpreting standard backpropagation literally as nervous-system learning. First, the hidden error requires an accurately weighted reverse signal, including the same effective forward weights; this weight-transport requirement is not supplied by the ordinary local Hebbian learning rule. Second, a supplied output target and differentiable layer-by-layer credit assignment idealize the feedback and temporal information available to biological neurons, which use spike trains, delays and local synaptic plasticity. Success at learning a task establishes computational usefulness, not that the nervous system implements the identical algorithm.

Use the four-input parity network. Let $S=z_1+z_2+z_3+z_4$. Four hidden units each receive all inputs with weight one and have thresholds $1/2,3/2,5/2,7/2$. Thuswhere $H(a)=1$ for $a\ge 0$ and zero otherwise. The output receives weights $(1,-1,1,-1)$ and has threshold $1/2$:The computation depends only on input count, so the following table checks all sixteen patterns at once:The output is one exactly for an odd input count. The hidden features detect nested count thresholds, whose alternating sum extracts odd parity.

This representation does not make learning automatically easy. Hard binary threshold units have no usable ordinary derivative; backpropagation normally uses smooth approximations. With uniformly sampled parity patterns, each individual input has zero correlation with the target, so simple first-order features provide no direct cue to the required interaction. The desired output also flips between neighboring count classes. A smooth network has sufficient capacity, but its training can be sensitive to initialization, saturation and local optimization; representability and trainability are different issues.

For one observation, let ${\ell }_i={\sum }_{c=0}^1{t}_{ic}\log p_{ic}$ and define ${\delta }_{ic}=t_{ic}-p_{ic}$. Differentiating the softmax function and its log-likelihood gives $\partial {\ell }_i/\partial o_{ic}={\delta }_{ic}$. The sigmoid-softmax network gradients areThese equations are backpropagation: use the chain rule through the output affine map, then ${\sigma }^{\prime }(u)=\sigma (u)(1-\sigma (u))$ through the hidden units, then the input affine map.

For a learning rate $\eta >0$, select one observation and update every parameter by $\theta \leftarrow \theta +\eta \nabla {\ell }_i(\theta )$, computing all entries of the gradient from the old parameters before applying the update. A uniformly selected observation gives an unbiased gradient of the average log-likelihood; multiplying by 3000 would instead estimate the gradient of the sum. Equivalently perform stochastic gradient descent on the negative log-likelihood. Batch size one makes one update per observation, and five epochs make five passes through the data, typically with a fresh random ordering on each pass.

For one-hot labels $y_{il}$ and predicted probabilities $p_{il}(\theta )$, the categorical cross-entropy loss isStochastic gradient descent initializes $\theta$, randomly orders the observations in each of five epochs, and for each single-observation batch computes a forward pass, the sample loss, and its gradient, then updatesBackpropagation is used after the forward loss evaluation to compute this gradient from the output layer back through the hidden layers.

A training epoch is conventionally one pass through the training data used by an iterative learning procedure. Several parameter updates can occur within that pass. A mini-batch supplies a group of examples for an update, while the epoch counts progress through the data collection rather than counting individual updates.^[ref]

Suppose there are $N=10$ training examples. Visit each exactly once per pass, form mini-batches of at most $B=4$ examples and retain the smaller final group. Their sizes are $4,4,2$. With one update for every mini-batch, an epoch hasupdates. Five epochs therefore make fifteen updates and fifty example visits. Fifty visits do not mean fifty distinct examples: the same ten examples are reused.

Changing the mini-batch size changes the update count. Taking all ten examples together gives one update per epoch under the same rule, while groups of two give five. An epoch count alone consequently does not specify the number of stochastic gradient descent steps, the amount of computation or the final fitted model.

The finite-pass assumption matters. Dropping an incomplete final group in this example gives two updates and only eight visits during that iterator traversal. Reordering the examples on later passes can change which examples are omitted. Sampling with replacement can repeat some examples before visiting others, so a chosen number of sampled updates need not be a literal once-per-example pass. Report those conventions rather than assuming that every loop named epoch guarantees full coverage.

The learning rate schedule needs a declared clock too: changing the learning rate once per epoch differs from changing it once per mini-batch. The optimization tutorial separates the outer loop from the operations that clear gradients, perform backpropagation and update parameters inside the training loop.^[ref] Keeping data passes and update steps distinct makes training runs easier to compare.

Exact backpropagation computes hidden teaching signals using the transpose of the current forward weight matrix. Explaining how a biological feedback pathway could know or maintain those weights is the weight transport problem. Useful approximate error correction need not satisfy exact reciprocal weight equality.^[ref]