Past exam of the how to build a brain from scratch course of the University of Oxford / 2022 / 2

Complementary learning systems address the conflict between rapidly remembering a particular experience and gradually learning shared structure without destroying prior knowledge. In the 1995 framework, the hippocampal system rapidly stores relatively separated event representations, while the neocortex learns more slowly through overlapping representations. Separation reduces interference between episodes; overlap allows related experiences to influence common features and support generalization. The hippocampal system can reinstate experiences for interleaved cortical learning. Thus the distinction is not simply a short-lived store versus a long-lived store, nor two copies differing only in learning rate. Representation, learning dynamics and communication between systems are complementary.^[ref]

The computational difficulty is catastrophic interference. When many memories share parameters, updates that improve a new task can impair previous tasks. Completely isolating every experience would reduce this problem, but would also prevent useful transfer through shared features. The intelligent agent therefore faces a stability-plasticity dilemma: it needs to remain responsive to new evidence while retaining useful old structure.

An original linear example exposes the issue without assuming contradictory facts. Let $f_w(x)=w^{\mathsf{T}}x$. An old example is $x_A=(1,0)$ with target $1$, already fitted by $w=(1,0)$. A new example is $x_B=(1,1)$ with target $0$. One gradient descent step on its squared error, with learning rate $1/2$, givesThe new prediction is now correct, but the old prediction falls from $1$ to $1/2$. Both examples could have been fitted exactly by $w_{\ast }=(1,-1)$, so the loss is due to the training history, not an impossible specification. Interleaving old and new examples instead estimates the gradient of their joint loss function,whose unique minimum is $w_{\ast }$. This small calculation illustrates interference and the benefit of retaining old constraints; it is not a reproduction of the published CLS simulations or measured forgetting.

A fast episodic store gives the intelligent agent immediate access to the new observation even while its slower predictive system is still learning. Subsequent retrieval supplies examples to be mixed with new experience. Over many updates, the slower system can extract regularities while accommodating exceptions. The mathematical target is a suitably weighted mixture of previous and current experience, not necessarily an exact photographic copy of every past event. Biased or incomplete replay can still distort learning, and finite storage does not eliminate the stability problem indefinitely.

The anatomical proposal fits previously established computations: dentate gyrus sparsification and pattern separation, recurrent hippocampal CA3 association, and hippocampal CA1 and cortical output pathways for reinstatement. These functions give a plausible route from a partial cue to a reactivated episode. They do not require every neuron or every memory to obey one perfectly separated code. A complementary architecture also need not imply that cortex never learns rapidly.

A motivating recording experiment compared hippocampal place-cell coactivity during spatial behaviour with sleep before and after it. Cell pairs coactive during behaviour were more likely to coactivate in subsequent sleep; inactive or spatially nonoverlapping pairs provided controls. This supports experience-related reactivation, rather than merely elevated firing during sleep. The 1994 result concerns ensemble coactivation: it should not be inflated into direct proof that a complete temporally ordered memory was transferred into cortex.^[ref]

Intervention adds a different kind of evidence. Selectively suppressing hippocampal sharp-wave ripples during post-training consolidation periods impaired a spatial memory task in rats. This supports a functional role for ripple-associated processing, but does not reveal which information was replayed or establish that cortical weight changes followed the exact interleaving algorithm.^[ref]

An important qualification comes from flavour–place learning. After rats had acquired a structured set of associations, new information fitting that schema could be consolidated much more rapidly. This challenges an account in which cortical integration always requires the same long delay. It is compatible with the broader computational idea that information consistent with an established representation can be incorporated with less disruptive change.^[ref]

For artificial intelligent agents, an episodic memory and experience replay can similarly support rapid use of new observations and continued training of shared representations. This is a functional analogy, not proof that a replay buffer reproduces the hippocampus. It needs appropriate retrieval, protection against sampling bias and a representation that preserves relevant context. Exact episode details, semantic regularities and current goals may require different readouts.

The central contribution of complementary learning systems is a computational explanation of why rapid episodic storage and overlapping generalization should cooperate. The theory predicts distinctive effects of learning history, replay disruption and prior knowledge, while its biological implementation and the eventual hippocampal dependence of detailed memories remain matters for further testing.