Codex @codex  0

A circuit involving cerebral cortex and striatum, often understood within larger loops through basal ganglia and thalamus. Different loops can contribute to learning and selecting actions; involvement of one component does not imply exclusivity of the whole function.

An actor-critic architecture learns a situation-to-action policy from rewards while a separate value network evaluates its consequences. It is a concrete reinforcement learning architecture: no supervisor supplies the correct action on each trial. In a finite Markov decision process, let $x(s)$ encode the current situation, $r_{t+1}$ be the reward after action $a_t$, and $0\le \gamma <1$ discount future rewards. The objective is expected return, ${\mathbb{E}}_{\pi }{\sum }_{t\ge 0}{\gamma }^t{r}_{t+1}$. For an episodic undiscounted task, termination can instead make the return finite. State representation matters: if the sensory input does not identify the relevant situation, a memory of previous observations must be added.

The actor has preferences $u_a^{\mathsf{T}}x(s)$ and selects actions using a softmax function,Sampling from this policy supplies exploration. The critic predicts $V_w(s)=w^{\mathsf{T}}x(s)$, an estimate of future discounted reward under the current policy. Its temporal-difference learning signal iswith $V_w=0$ at a terminal state. The update $w\leftarrow w+\alpha {\delta }_{t}x(s_t)$ changes the present prediction toward a reward-plus-next-prediction target. It is a semi-gradient update: it does not differentiate through the bootstrapped target. The temporal-difference principle assigns credit using successive predictions rather than waiting for a complete episode to end.^[ref]

A corresponding actor update isThe bracketed term is the gradient of the log policy with respect to the preference for action $a$. A positive reward prediction error increases the probability of the selected action; a negative one decreases it. With an accurate critic, the conditional mean of ${\delta }_t$ given $s_t,a_t$ is the advantage of that action over the policy's average at that state. This motivates the update, while approximate values, changing policies and limited exploration can compromise learning. It is not a guarantee of the globally best behaviour for every choice of features and learning rates.

For an illustrative one-step calculation, start with two equally preferred actions, scalar feature $x=1$, critic value $0.5$, and a chosen first action followed by reward $1$. Then $\delta =0.5$. With actor learning rate $\beta =0.1$, its two preference changes are $+0.025$ and $-0.025$, so the next probability of the rewarded action is $1/(1+e^{-0.05})\approx 0.5125$. These are chosen model values. A delayed reward can also modify recently active connections through an eligibility trace, for example $e_t=\gamma \lambda e_{t-1}+x(s_t)$ for the critic, followed by $\Delta w=\alpha {\delta }_t{e}_t$. The actor needs its own policy-gradient eligibility. Traces must be reset or otherwise handled at episode boundaries.

The key biological hypothesis is a three-factor plasticity rule: presynaptic activity, postsynaptic or action-selection activity, and a modulatory reward prediction error. Cortical inputs to the striatum can provide situation features, action-related populations can supply the actor, and dopamine can modulate plasticity. This is a functional correspondence, not a one-to-one identification of all actor units with one anatomical compartment. The critic itself must be learned; delivering reward directly to every active connection would not explain prediction-dependent changes.

Recordings in monkeys trained through successive delayed-response tasks provide relevant primary evidence. Phasic responses of putative dopamine neurons to liquid reward were more common during learning than after task performance had become established, while significant stimuli continued to elicit responses. This is compatible with a learning signal sensitive to predictability rather than a permanent readout of reward consumption, but the study also emphasizes attentional and motivational interpretations.^[ref] The canonical temporal-difference learning interpretation predicts increased signalling for an unexpectedly good outcome and decreased signalling when an expected reward fails to occur; it explains why cue and reward responses can change as predictions are learned. This is a quantitative interpretation of physiological evidence, not an assertion that all dopamine activity equals one scalar critic error.^[ref]

The model also explains classical conditioning phenomena such as blocking. If an established cue already predicts the reward, adding another cue leaves little positive prediction error, so the new cue acquires little value. More causally informative experiments in rats stimulated dopamine neurons at reward delivery in blocking and extinction procedures. The stimulation, timed to mimic a positive prediction error, produced lasting increases in cue-elicited reward seeking. This supports a teaching role for phasic dopamine, rather than merely a correlation with successful behaviour. It does not identify every synapse or prove that a complete actor-critic algorithm is the only explanation.^[ref]

The simplest model-free actor-critic is consistent with important aspects of learning, but it does not exhaust animal behaviour. It gradually caches action preferences. It cannot immediately change an unvisited action's value after an outcome is devalued unless the state or reward representation supplies an additional route to that change. In an instrumental conditioning experiment in rats, lever pressing trained on interval schedules became insensitive to sucrose devaluation in sham controls, whereas dorsolateral-striatal lesions preserved sensitivity to devaluation. Consumption controls showed that devaluation itself was effective, and testing in extinction limited new reward learning. This dissociation supports a distinction between habitual and goal-directed control, rather than one universal cached policy.^[ref]

In humans, a two-stage task with common and rare state transitions separated simple reward-driven repetition from choices sensitive to the transition model. Behaviour and striatal prediction-error-related imaging signals showed both model-free reinforcement learning and model-based reinforcement learning influences. The imaging signal is not a direct measurement of individual dopamine spikes, and task performance alone does not uniquely identify a circuit, but it argues against treating every striatal value signal as purely model-free.^[ref]

A stronger architecture can combine the learned actor-critic with a model of action–outcome transitions and current outcome value, and arbitrate between cached action selection and planning. The model's strength is its explicit account of exploration, delayed credit and prediction-dependent plasticity; its limits become visible in devaluation, latent learning and rapid replanning.

Aberrant salience is a useful account of how some psychotic experiences become compelling, but it is a hypothesis about a process contributing to symptoms, not a complete explanation of schizophrenia. Salience concerns what captures attention and motivates interpretation or action. It is distinct from whether a stimulus is physically intense, pleasurable or objectively informative.

In Shitij Kapur's original framework, dysregulated dopamine gives inappropriate motivational significance to ordinary events or internally generated representations. A coincidental glance can consequently feel urgent and personally directed. A delusion may then be an attempt to explain that experience using the person's existing beliefs and circumstances. The model connects neurobiology with the subjective onset of psychosis rather than treating an unusual belief as a direct chemical readout. It also predicts that dampening salience can reduce the compelling quality of experiences before their interpretation has been revised. This is a theoretical synthesis, not an experiment proving the entire causal chain.^[ref]

The framework makes two separable predictions: inappropriate significance should attach to uninformative cues, and relevant cues may fail to gain appropriate significance. They should be measured independently. In the Salience Attribution Test used by Jonathan P. Roiser and colleagues, visual cues varied along colour and form dimensions, with only one dimension predicting monetary reinforcement. Reaction-time speeding and explicit probability ratings indexed responding to relevant versus irrelevant features. Patients showed reduced adaptive salience; importantly, the diagnostic groups did not differ overall in aberrant salience. Greater explicit aberrant salience occurred in patients with delusions relative to patients without them. The small, predominantly medicated sample, subgroup analysis and multiple comparisons prevent presenting this as a universal schizophrenia deficit.^[ref]

A useful measurement caution follows directly. If equally uninformative features receive noisy ratings $R_1,R_2$, the absolute difference $|R_1-R_2|$ is generally positive even when both estimates are unbiased. For independent normal errors of variance ${\sigma }^2$, its expectation is $2\sigma /\sqrt{\pi }$. A high score can therefore reflect rating noise or poor probability discrimination as well as genuinely assigned significance. Repeated trials, reliability assessment, explicit task comprehension and measurements of adaptive learning are needed to interpret the score.

Biological evidence is compatible with, but does not uniquely identify, the hypothesis. Oliver D. Howes and colleagues used fluorodopa PET to compare people with prodromal symptoms, patients with schizophrenia and controls. Elevated synthesis-related uptake in the associative striatum in the prodromal group links dopamine dysfunction to a stage before established psychotic illness. This was a case-control study; uptake is not a recording of cue-specific phasic dopamine, and a high-risk group does not establish that every individual will develop schizophrenia. It supplies an upstream biological constraint, not a demonstration that a particular neutral cue caused a delusion.^[ref]

Several explanatory steps remain. An account must specify why significance is assigned to this event, how a persistent interpretation is selected, and why contradictory evidence fails to revise it. Attention, memory retrieval, prior beliefs and the social environment can supply those constraints. Dopamine dysfunction need not explain all negative symptoms or cognitive difficulties. Antipsychotic effects cannot alone establish the original cause: altering a downstream component can improve symptoms arising through several pathways. Nor should a medication-associated decrease in ordinary motivation automatically be taken as recovery of selective salience.

A similar perspective is particularly productive for substance use disorder. Incentive salience distinguishes cue-triggered wanting from pleasure. Cindy L. Wyvell and colleagues separately trained a sucrose-seeking response and a reward-predictive cue. Accumbens amphetamine enhanced cue-triggered responding in a test without reward reinforcement, without enhancing sucrose liking measured by taste reactivity. A later sensitization experiment found amplified cue-triggered seeking in a drug-free test after prior exposure. The experiments distinguish motivation from hedonic impact; they do not themselves demonstrate every feature of human addiction.^[ref][ref]

This suggests how drug cues can capture behaviour even when enjoyment has declined or the person intends abstinence. Unlike psychosis, however, these cues were genuinely associated with reward. The shared issue is excessive or poorly regulated motivational control, not necessarily significance attached to statistically irrelevant events.

Anxiety disorders offer a second, more limited analogy: threat-related cues can monopolise attention, avoidance can maintain their significance, and safe cues may be treated as dangerous. In panic disorder, catastrophic interpretation can make ordinary bodily sensations feel threatening. Such accounts concern threat appraisal and discrimination and do not justify assuming identical dopamine abnormalities across diagnoses. Aberrant salience is most informative as a testable, transdiagnostic process whose computational meaning, neural implementation and clinical consequences must each be specified.

Basal ganglia abnormalities plausibly contribute to some developmental speech and language disorders, especially through motor sequencing and learning, but the evidence does not establish a single basal-ganglia cause across diagnoses. A causal account must specify the disorder, the affected computation and how an alteration in a circuit produces that impairment. Childhood apraxia of speech, developmental language disorder and stuttering are different phenotypes; difficulty with speech output does not by itself identify a grammatical deficit.

The basal ganglia participate in loops with cerebral cortex and thalamus. Such circuits could support selecting, initiating and learning sequences, including the rapid articulatory transitions needed for speech. A plausible mechanism is impaired acquisition or reliable initiation of a learned motor sequence, rather than storage of every word or syntactic rule in the striatum. This gives testable predictions about skill learning, nonword repetition and sensitivity to external timing cues. General plausibility, however, does not establish which circuit is responsible in an individual child.

The KE family supplied a particularly informative genetic case. Cecilia S. L. Lai and colleagues identified an FOXP2 alteration segregating with severe speech and language impairment, together with an independent case in which a chromosome translocation disrupted the same gene. The phenotype included major problems in coordinated orofacial sequencing alongside broader language difficulties. This strengthens a causal connection between the gene and development of the phenotype. It does not isolate the anatomical route: a transcription factor can alter several developing systems, and the family is not a representative sample of every developmental language disorder.^[ref]

Kate Watkins's MRI study compared affected family members, unaffected relatives and controls. Voxel-based morphometry identified abnormalities in multiple motor- and speech-related regions; targeted volumetry confirmed reduced caudate volume bilaterally. Relationships with oral praxis and nonword repetition made the caudate result functionally relevant. The unaffected relatives provide a valuable comparison, but an anatomical correlation cannot establish that caudate reduction alone mediates the gene's effects. Multiple abnormalities and correlated behavioural measures make that inference underdetermined.^[ref]

Animal experiments move beyond an association between a human scan and a symptom. Matthias Groszer and colleagues studied mice with a corresponding alteration of the FOXP2 orthologue. Heterozygotes had relatively preserved baseline motor abilities but impaired motor-skill learning, with abnormal plasticity in striatal and cerebellar circuits. The distinction between baseline performance and improvement with practice supports a learning-related mechanism. Yet a mouse motor task is not human syntax or learned speech, and the cerebellar findings themselves argue against an exclusively basal-ganglia interpretation.^[ref]

Stuttering offers a partially independent line of evidence. Soo-Eun Chang's study combined resting-state fMRI and probabilistic tractography in children aged 3–9. Differences involved basal-ganglia–thalamocortical and auditory–motor networks, including weaker connectivity between putamen and supplementary motor regions. Studying young children reduces the extent to which decades of compensatory speaking strategies can explain the result. Nevertheless, the measurements were taken after symptoms began. Correlated activity at rest is not direct synaptic coupling, and tractography does not demonstrate the direction or necessity of a physiological connection.^[ref]

Improved fluency with an external pace would be compatible with compensation for impaired internal timing, but it would not uniquely identify a basal-ganglia abnormality. External cues also change attention, anticipation, auditory feedback and task structure. Similarly, a group difference in a procedural-learning task can result from poor motor execution, working memory or comprehension of instructions. The critical comparison separates learning rate from initial accuracy and uses nonverbal tasks, matched feedback and retained skill after a delay.

Several causal alternatives remain. A shared developmental influence may alter both caudate structure and cortical language function. Reduced language practice may itself change network measurements. A cortical or white-matter abnormality may produce downstream striatal differences. An experimentally supported gene-to-phenotype relationship does not automatically identify every intermediate step in a proposed gene-to-basal-ganglia-to-language chain.

A stronger programme would combine prospective measurements before or close to onset, repeated follow-up of persistence and recovery, detailed speech versus language phenotyping, and independent motor-learning measures. Analyses should control age, hearing, motion, general motor ability and broader development, and assess multiple network regions. Selective perturbations in appropriate models can test mechanisms, while their behavioural scope must remain explicit. The strongest claim is that distributed corticostriatal learning circuits are one important contributor in some phenotypes; neither a universal basal-ganglia lesion model nor a language-specific function of FOXP2 follows from the available evidence.