Past exam of the how to build a brain from scratch course of the University of Oxford / 2022 / 5 / Solution

A world model supports intelligent behaviour by predicting the consequences of actions that need not be executed first. The prefrontal cortex can contribute task-state and goal control, while the hippocampus can supply relational and episodic content for constructing possible situations and trajectories. This is a division of computational requirements, not a rigid one-region-per-function scheme.

In a Markov decision process, a model can represent transitions $\hat{P}(s^{\prime }\mid s,a)$ and expected rewards $\hat{r}(s,a)$. Planning uses them to evaluate an action through a backup such asLonger searches apply the same principle across alternative trajectories. A world model can be symbolic, distributed, probabilistic or a combination; it need only preserve information relevant to the decision. In a partially observed situation, the intelligent agent must infer a state or maintain uncertainty over states rather than treating the latest image as a complete description of the world.

For a simple constructed example, suppose one action leads to food and another to water, with current outcome values $2$ and $1$. After the food is devalued to $0$, an intelligent agent that knows the action–outcome relations can immediately prefer water without experiencing the food-producing action again. A purely cached action value does not necessarily change at that moment. This demonstrates the usefulness of an internal model; it does not imply that every devaluation effect uniquely identifies one algorithm, because inferred state changes or offline value updates can also alter cached decisions.

Mental simulation can recombine remembered components into a possible future, evaluate an untried route and compare goals. It differs from simply replaying a previous trajectory verbatim. It also differs from episodic memory retrieval: simulated content may describe something that never happened. Both operations can use shared representations, so a new imagined situation need not be generated independently of memory.

The prefrontal cortex is well placed to contribute the control side of this process: maintaining the current task, selecting relevant information, representing goals and using prospective outcomes to guide action. It is heterogeneous, and activity in one subregion is not evidence that all of prefrontal cortex carries a complete internal simulator. A particularly relevant hypothesis concerns the orbitofrontal cortex: a useful decision state may depend on previous events and task rules even when current sensory input is unchanged.

A human fMRI study used superimposed face and house images with a rule determining which category to judge from the preceding task history. Different hidden task states could therefore accompany similar current inputs. Cross-validated multivariate decoding recovered task-state information from orbitofrontal cortex, and decoding fidelity related to behavioural accuracy. This supports a role in representing the situation in which values and actions are evaluated. It does not demonstrate that the decoded patterns themselves contain a complete future trajectory, and noninvasive imaging cannot identify the synaptic algorithm.^[ref]

The hippocampus provides a complementary source of spatial and relational content. A cognitive map describes relationships among locations or task states rather than only a sequence of rewarded responses. Such a representation can support a novel route when goals change, and event representations can contribute objects, locations and contextual relations to a simulated scene. Whether an internally generated sequence represents a possible future, a remembered past or a training signal must be tested rather than inferred from sequence-like firing alone.

In freely moving rats choosing journeys between food locations, hippocampal place-cell ensembles sometimes produced internally generated sequences before travel. Decoded sequences described trajectories related to remembered goals and predicted upcoming behaviour. The ordering and timing make the result relevant to prospective computation, but the recording is correlational: a sequence could reflect a prepared choice or goal-related retrieval rather than perform the decisive search itself.^[ref]

A separate intervention interrupted awake sharp-wave ripples during a spatial alternation task and impaired learning and performance. Place fields and post-experience reactivation were relatively preserved, helping distinguish the effect from wholesale destruction of spatial coding or later replay. This supports a causal contribution of awake ripple-associated processing to memory-guided decisions, without identifying every disrupted sequence as a planning calculation.^[ref]

Human lesion evidence extends the discussion beyond navigation. Patients with predominantly bilateral hippocampal damage were asked to construct new experiences from short scenario cues. Their descriptions were generally less spatially coherent than those of matched controls, despite preserved semantic retrieval; one patient performed well. The task therefore supports a hippocampal contribution to binding imagined scene elements, while the spared case, lesion selectivity and the difference between scene construction and action planning limit a universal necessity claim.^[ref]

A plausible integrated model lets prefrontal representations specify the task and current goal, hippocampal and cortical representations generate or retrieve candidate situations, and valuation and action-selection circuits compare their consequences. Communication can be iterative: candidate outcomes change what information is sought, and new observations update the inferred state. This is more specific than assigning “imagination” to the hippocampus and “intelligence” to prefrontal cortex, but remains a hypothesis about their interaction.

Planning also has costs. Long simulations can amplify model errors, and unlimited search is computationally expensive. An intelligent agent should select a useful abstraction, prioritize relevant branches and decide when a cached policy is adequate. Tests should distinguish sensitivity to reward changes from sensitivity to transition changes, and compare neural predictions before a decision rather than only after a successful action. World models enable flexible prospective control; the evidence implicates interacting state, memory and valuation processes, rather than a single anatomical seat of mental simulation.