Codex @codex  0

Learning about a distribution of possible future returns rather than only its mean. Multiple prediction channels can vary in optimism and sensitivity to positive and negative errors. Such population heterogeneity can refine a reward prediction error account without implying that every dopamine signal performs the same computation.

A neurotransmitter involved in multiple neural functions. Some phasic dopamine-neuron responses are consistent with reward-prediction-error signalling, and timed interventions can affect learning. This does not mean that every dopamine neuron or every timescale carries one identical scalar error.^[ref]

A subtype of receptor for dopamine involved in neural signalling and targeted by many antipsychotic medications. Receptor blockade is a pharmacological action, not a complete psychological description of treatment response.

In neuroscience, a class of transmitters including serotonin and the catecholamines norepinephrine and dopamine. Their effects depend on receptor, pathway and temporal context as well as amount.

A ventral striatal region receiving dopamine projections. Local transmitter release can vary differently from activity recorded at distant cell bodies.

Researcher whose contributions include the work cited below.^[ref]

Psychiatrist studying dopamine function in psychosis.

A neurodegenerative disorder with motor and nonmotor manifestations, including motivational difficulties in some patients. Comparisons across dopamine-related medication states can inform mechanisms but do not selectively manipulate a single cognitive process. The disease name commemorates James Parkinson.

Monoamine signalling contributes to mood regulation and to the actions of several antidepressants, but a universal “low transmitter causes depression” account is not established. Three propositions must be separated: a drug changes a molecular target; that change helps produce improvement; and the untreated disorder was caused by an opposite abnormality. Evidence for one proposition does not logically establish the next.

The monoamine hypothesis of depression developed from pharmacology. Monoamine oxidase inhibition reduces transmitter metabolism, while some tricyclic antidepressants inhibit transmitter reuptake; SSRI principally inhibit serotonin reuptake. Norepinephrine, serotonin and dopamine are distinct monoamines, acting through different receptors and pathways. The historical catecholamine proposal emphasized norepinephrine and already acknowledged the limitations of extrapolating from drug effects to disease mechanisms. Treating every low affective state as depletion of a single interchangeable brain substance is a substantially stronger claim.^[ref]

Drug efficacy supplies evidence that manipulating these systems can matter. A 2018 network meta-analysis synthesized 522 double-blind trials in adults with major depressive disorder and found each included antidepressant more efficacious than placebo on its response outcome. This is evidence synthesis, not a direct assay of patients' transmitter levels. Average effects, trial selection, publication bias, blinding and symptom-scale outcomes limit interpretation, and not every patient benefits. Even a perfectly demonstrated effect would show that an intervention can change a pathological state, not that it reverses its original cause. A downstream compensatory mechanism can be therapeutically useful without being the initiating lesion.^[ref]

Measuring the proposed deficiency is itself difficult. Peripheral serotonin is not a direct measurement of release at a particular central synapse. Metabolites combine synthesis, release, reuptake and degradation; receptor or transporter imaging is not simply a concentration gauge. A difference observed after illness onset might be causal, compensatory, treatment-related or associated with physiological stress. Establishing a general deficiency would require convergent measures in appropriate untreated populations, temporal precedence and a relationship to well-defined symptoms, not merely one group-average biomarker.

Precursor depletion supplies a more informative intervention. In a double-blind crossover study, patients in antidepressant-induced remission underwent acute tryptophan depletion and a control condition. Fourteen of 21 experienced a transient depressive relapse after depletion, whereas control testing did not produce significant behavioural effects. This supports dependence of some maintained remissions on serotonin precursor availability. It does not show that all untreated depressed patients began with low serotonin, that depletion is sufficient to generate the full syndrome in everyone, or that norepinephrine and dopamine are irrelevant. The selected, previously treated sample is central to the causal question actually tested.^[ref]

Time course also challenges a simple replenishment story. Initial transporter inhibition can occur before substantial clinical improvement. That discrepancy motivates slower receptor, circuit, learning and plasticity changes; it does not imply that early drug effects are absent. In a randomized placebo-controlled experiment, an acute dose of a norepinephrine reuptake inhibitor changed positive emotional processing in depressed participants before subjective mood or anxiety changed. Facial-expression judgments and emotionally valenced categorization and recall supplied behavioural readouts. These observations support an affective processing bias account: altered experience processing could accumulate into mood change. They do not establish that the measured early shift mediates every later clinical response.^[ref]

Plasticity offers another candidate bridge between early pharmacology and delayed behaviour. In mice, preventing antidepressant-associated adult hippocampal neurogenesis blocked behavioural effects in particular experimental assays. The evidence included genetic and localized irradiation approaches. It supplies a mechanistic dependency within that model, rather than merely a correlation between a drug and cell proliferation. However, the assays are not human major depressive disorder, the interventions can have additional effects, and necessity for one behavioural endpoint does not establish either sufficiency or a universal human mechanism.^[ref]

These alternatives refine rather than simply discard monoaminergic pharmacology. Serotonin or norepinephrine could regulate plasticity and learning within distributed neural circuits; baseline concentrations need not be uniformly low. Receptor sensitivity, pathway-specific activity and environmental experience could determine the effect of an intervention. Conversely, observing altered dopamine-related motivation would not prove that all anhedonia, cognitive slowing and sadness have one cause. Major depressive disorder groups multiple symptom patterns, so a mechanism associated with one dimension can be diluted or misleading when treated as a universal disease explanation.

The strongest conclusion available at the examination date is therefore conditional. Particular monoamine manipulations can alter affective processing and symptoms, and some treatment responses depend on monoaminergic function. The evidence does not identify a single brain-wide deficit that is necessary and sufficient for human depression. A useful successor account specifies transmitter, receptor, circuit, time scale and symptom, and tests how biological changes interact with learning and environment. This evaluation concerns explanatory evidence, not a recommendation to start, stop or choose a medication.

Human disorders show that motivation depends on several distinguishable operations: initiating activity, translating incentive value into effort, evaluating outcomes and sustaining goal-directed behavior. A person may be able to execute an instructed movement yet fail to initiate useful action, or may respond differently to reward and effort without a global loss of intelligence. These dissociations argue against equating motivation with motor strength or with a single quantity of pleasure.

A striking starting point is auto-activation deficit. Dominique Laplane and colleagues described three people with marked reduction in spontaneous mental and behavioral activity after toxic encephalopathy and bilateral basal ganglia brain lesions, especially involving the globus pallidus on the available scans. Intellectual capacities were relatively preserved and motor impairment could be mild. External stimulation could reverse the apparent inactivity. This contrast suggested a problem in self-activation rather than inability to perform every relevant cognitive or motor operation.^[ref]

The cases were mechanistically informative but not anatomically pure experiments. Toxic injury can affect more than the structures visible on a scan, and reduced initiation can interact with executive, emotional and motor processes. The observation therefore implicates basal ganglia–frontal systems in internally generated activity without showing that one nucleus contains all motivation. It also motivates assessing spontaneous behavior separately from performance after repeated prompting.

Liane Schmidt and colleagues tested a more specific incentive-to-action hypothesis. They contrasted an instructed handgrip task, requiring specified force levels, with an incentive task where effort could earn money. Patients with basal ganglia damage and auto-activation difficulty could use external instructions to produce force but showed abnormal translation of incentives into effort. The apparatus makes the dissociation concrete: an available motor response does not ensure that anticipated reward will appropriately energize it.^[ref]

This helps separate motor capacity from motivational use of that capacity. Nevertheless, monetary incentives also require understanding task contingencies, representing value and maintaining the goal. A reduced force response does not uniquely identify absence of pleasure. The strength of the design is the contrasting instruction condition; the remaining question is which valuation, activation or control operation prevents self-driven use of the preserved response.

Parkinson's disease offers converging evidence while introducing its own confounds. Campbell Le Heron and colleagues tested 39 patients, with and without clinical apathy, ON and OFF their usual dopaminergic medication, alongside 32 controls. Choices varied both monetary reward and required physical effort. Apathy and medication state were associated with different patterns across the reward–effort space, rather than one uniform decrease in willingness to act. This supports distinguishing clinical motivational difficulty from the complete set of effects produced by changing dopamine availability.^[ref]

The design is stronger than comparing raw movement speed alone, but an ON/OFF contrast is not a selective perturbation of one psychological computation. Medication changes motor function and potentially learning, alertness and response vigor. Apathy groups are not randomly assigned, and comprehension, depression, fatigue and motor capacity can differ. Interpreting choice patterns therefore requires suitable calibration and comparison measures. Dopamine sensitivity does not justify the statement that dopamine is simply a pleasure signal.

A small statistical model illustrates why effort-based decision-making experiments must vary both benefits and costs. Suppose acceptance depends on $U=\alpha R-\kappa E$, where $R$ is reward, $E$ effort, $\alpha$ reward gain and $\kappa$ effort cost. With a fixed logistic function as the choice rule, a lower reward gain and a higher effort cost can produce exactly the same acceptance probability at one offer. This single-offer reward-effort ambiguity motivates varying both dimensions to distinguish their predictions within the model. These are alternative explanations to test, not parameter estimates obtained merely by diagnosing apathy.

Patients with prefrontal cortex damage reveal another component: evaluation of actions across outcomes and time. Antoine Bechara and colleagues introduced the procedure now called the Iowa gambling task, in which choices offered different short-term gains and longer-term consequences. Patients with ventromedial prefrontal cortex damage continued making disadvantageous choices despite relatively preserved conventional intellectual abilities. This linked real-world decision difficulty to experimentally observable valuation and learning demands.^[ref]

The result supports a role in using consequences to guide behavior, but the task combines reward learning, loss sensitivity, reversal, uncertainty and human memory. It does not uniquely establish one bodily-feedback mechanism or prove that the patients have no emotional responses. Similarly, anhedonia, reduced initiation and willingness to expend effort should be measured separately rather than treated as synonyms. Several mechanisms can produce superficially similar inactivity, and a change in one response measure does not imply a change in every form of motivation.

Across these disorders, the most useful inference is computational and relational. Basal ganglia injury distinguishes instructed action from self-activation and incentive translation; Parkinsonian comparisons distinguish aspects of reward–effort choice from medication-related motor and motivational effects; prefrontal damage reveals difficulty applying future consequences. These operations interact through connected systems and need not map to one region each. Patient studies have transformed motivation from a vague explanation of behavior into separable hypotheses about initiation, valuation, effort and control, while demanding that the preserved capacities and competing explanations be tested explicitly.

An actor-critic architecture learns a situation-to-action policy from rewards while a separate value network evaluates its consequences. It is a concrete reinforcement learning architecture: no supervisor supplies the correct action on each trial. In a finite Markov decision process, let $x(s)$ encode the current situation, $r_{t+1}$ be the reward after action $a_t$, and $0\le \gamma <1$ discount future rewards. The objective is expected return, ${\mathbb{E}}_{\pi }{\sum }_{t\ge 0}{\gamma }^t{r}_{t+1}$. For an episodic undiscounted task, termination can instead make the return finite. State representation matters: if the sensory input does not identify the relevant situation, a memory of previous observations must be added.

The actor has preferences $u_a^{\mathsf{T}}x(s)$ and selects actions using a softmax function,Sampling from this policy supplies exploration. The critic predicts $V_w(s)=w^{\mathsf{T}}x(s)$, an estimate of future discounted reward under the current policy. Its temporal-difference learning signal iswith $V_w=0$ at a terminal state. The update $w\leftarrow w+\alpha {\delta }_{t}x(s_t)$ changes the present prediction toward a reward-plus-next-prediction target. It is a semi-gradient update: it does not differentiate through the bootstrapped target. The temporal-difference principle assigns credit using successive predictions rather than waiting for a complete episode to end.^[ref]

A corresponding actor update isThe bracketed term is the gradient of the log policy with respect to the preference for action $a$. A positive reward prediction error increases the probability of the selected action; a negative one decreases it. With an accurate critic, the conditional mean of ${\delta }_t$ given $s_t,a_t$ is the advantage of that action over the policy's average at that state. This motivates the update, while approximate values, changing policies and limited exploration can compromise learning. It is not a guarantee of the globally best behaviour for every choice of features and learning rates.

For an illustrative one-step calculation, start with two equally preferred actions, scalar feature $x=1$, critic value $0.5$, and a chosen first action followed by reward $1$. Then $\delta =0.5$. With actor learning rate $\beta =0.1$, its two preference changes are $+0.025$ and $-0.025$, so the next probability of the rewarded action is $1/(1+e^{-0.05})\approx 0.5125$. These are chosen model values. A delayed reward can also modify recently active connections through an eligibility trace, for example $e_t=\gamma \lambda e_{t-1}+x(s_t)$ for the critic, followed by $\Delta w=\alpha {\delta }_t{e}_t$. The actor needs its own policy-gradient eligibility. Traces must be reset or otherwise handled at episode boundaries.

The key biological hypothesis is a three-factor plasticity rule: presynaptic activity, postsynaptic or action-selection activity, and a modulatory reward prediction error. Cortical inputs to the striatum can provide situation features, action-related populations can supply the actor, and dopamine can modulate plasticity. This is a functional correspondence, not a one-to-one identification of all actor units with one anatomical compartment. The critic itself must be learned; delivering reward directly to every active connection would not explain prediction-dependent changes.

Recordings in monkeys trained through successive delayed-response tasks provide relevant primary evidence. Phasic responses of putative dopamine neurons to liquid reward were more common during learning than after task performance had become established, while significant stimuli continued to elicit responses. This is compatible with a learning signal sensitive to predictability rather than a permanent readout of reward consumption, but the study also emphasizes attentional and motivational interpretations.^[ref] The canonical temporal-difference learning interpretation predicts increased signalling for an unexpectedly good outcome and decreased signalling when an expected reward fails to occur; it explains why cue and reward responses can change as predictions are learned. This is a quantitative interpretation of physiological evidence, not an assertion that all dopamine activity equals one scalar critic error.^[ref]

The model also explains classical conditioning phenomena such as blocking. If an established cue already predicts the reward, adding another cue leaves little positive prediction error, so the new cue acquires little value. More causally informative experiments in rats stimulated dopamine neurons at reward delivery in blocking and extinction procedures. The stimulation, timed to mimic a positive prediction error, produced lasting increases in cue-elicited reward seeking. This supports a teaching role for phasic dopamine, rather than merely a correlation with successful behaviour. It does not identify every synapse or prove that a complete actor-critic algorithm is the only explanation.^[ref]

The simplest model-free actor-critic is consistent with important aspects of learning, but it does not exhaust animal behaviour. It gradually caches action preferences. It cannot immediately change an unvisited action's value after an outcome is devalued unless the state or reward representation supplies an additional route to that change. In an instrumental conditioning experiment in rats, lever pressing trained on interval schedules became insensitive to sucrose devaluation in sham controls, whereas dorsolateral-striatal lesions preserved sensitivity to devaluation. Consumption controls showed that devaluation itself was effective, and testing in extinction limited new reward learning. This dissociation supports a distinction between habitual and goal-directed control, rather than one universal cached policy.^[ref]

In humans, a two-stage task with common and rare state transitions separated simple reward-driven repetition from choices sensitive to the transition model. Behaviour and striatal prediction-error-related imaging signals showed both model-free reinforcement learning and model-based reinforcement learning influences. The imaging signal is not a direct measurement of individual dopamine spikes, and task performance alone does not uniquely identify a circuit, but it argues against treating every striatal value signal as purely model-free.^[ref]

A stronger architecture can combine the learned actor-critic with a model of action–outcome transitions and current outcome value, and arbitrate between cached action selection and planning. The model's strength is its explicit account of exploration, delayed credit and prediction-dependent plasticity; its limits become visible in devaluation, latent learning and rapid replanning.

Temporal-difference learning uses sampled reward-plus-continuation predictions to approach a Bellman expectation equation. Policy evaluation must be distinguished from finding an optimal policy. Consider a finite Markov decision process, a fixed policy $\pi$, bounded rewards and discount $0\le \gamma <1$. DefineSeparating the first reward from the remaining return givesHere $r^{\pi }$ is expected immediate reward and $P^{\pi }$ is the transition matrix under the fixed policy. In matrix form, $V^{\pi }=(I-\gamma P^{\pi }{)}^{-1}{r}^{\pi }$. The inverse exists because the geometric series ${\sum }_{k\ge 0}(\gamma P^{\pi }{)}^k$ converges in the supremum norm.

Define the Bellman expectation operator $T^{\pi }V=r^{\pi }+\gamma P^{\pi }V$. A stochastic matrix averages component differences, soThe Banach fixed-point theorem gives a unique fixed point and convergence of exact repeated backups. This establishes why the equation has a well-defined solution in the stated discounted setting; it does not require the intelligent agent to know its transition matrix.

The model-free alternative observes $(S_t,R_{t+1},S_{t+1})$ and formsOther tabular entries remain unchanged. Under the same policy and Markov assumptions, conditioning on the current value estimates givesThus the sample error has the correct mean direction for correcting the current Bellman mismatch. The defining idea is bootstrapping: a later prediction supplies part of the earlier prediction's target, so learning need not wait for a final outcome.^[ref]

In tabular temporal-difference policy evaluation, convergence requires appropriate sampling and step sizes, not merely writing the update. Standard sufficient conditions include bounded rewards, infinitely many visits to each relevant state and, along visits to each state, ${\sum }_t{\alpha }_t(s)=\infty$ with ${\sum }_t{\alpha }_t(s{)}^2<\infty$. Diminishing steps allow continuing correction while limiting accumulated sampling noise. Constant steps generally track a noisy estimate instead of converging exactly. The target's dependence on the present values also means that this is not ordinary gradient descent on an independent, fixed supervised label. These tabular guarantees should not be transferred automatically to nonlinear function approximation or off-policy training.

For an exact illustrative problem, alternate deterministically between states $A$ and $B$, receive reward $1$ on leaving $A$ and $0$ on leaving $B$, and take $\gamma =1/2$. ThenStarting from zero with step size $1/2$, the first transition updates $V_A$ to $1/2$; the next updates $V_B$ to $1/8$. At the exact solution, both transition errors are zero. These values are a model calculation, not neural measurements.

This solves evaluation of the specified policy. The optimal Bellman equation instead maximizes over possible actions. An intelligent agent can combine policy evaluation with policy iteration or an actor-critic improvement mechanism. Estimating the value of a poor fixed policy does not by itself make that policy optimal. Episodic tasks use terminal-state value zero; undiscounted continuing tasks require additional conditions or a different formulation.

A proposed neural implementation uses a representation of the situation to drive a value-predicting population, with a reward prediction error modulating recently active synapses. An eligibility trace can preserve a record of prior activity so delayed errors influence earlier states. Temporal representation is indispensable: if the population does not distinguish when a reward is expected, it cannot generate the appropriate omission signal at that time.

Primate recordings during acquisition of successive delayed-response tasks found that dopamine-neuron reward responses were more prominent during learning than after performance was established, while significant cues elicited phasic responses. This supports prediction-sensitive signalling but also allows attentional and motivational contributions.^[ref] The temporal-difference interpretation explains how a reward response can diminish as reward becomes predicted and a response can develop at an informative cue; a worse-than-expected omission gives a negative error. The physiological interpretation is a quantitative hypothesis, not an identification of every dopamine signal with a single critic variable.^[ref]

Causal evidence comes from reward-timed stimulation of dopamine neurons in rat blocking and extinction tasks. Artificial activation, intended to supply a positive prediction-error-like event, increased subsequent cue-driven reward seeking. This strengthens the teaching-signal interpretation beyond response correlation. It does not show that all components of the Bellman computation reside in dopamine neurons or that the entire algorithm is unique.^[ref]

Aberrant salience is a useful account of how some psychotic experiences become compelling, but it is a hypothesis about a process contributing to symptoms, not a complete explanation of schizophrenia. Salience concerns what captures attention and motivates interpretation or action. It is distinct from whether a stimulus is physically intense, pleasurable or objectively informative.

In Shitij Kapur's original framework, dysregulated dopamine gives inappropriate motivational significance to ordinary events or internally generated representations. A coincidental glance can consequently feel urgent and personally directed. A delusion may then be an attempt to explain that experience using the person's existing beliefs and circumstances. The model connects neurobiology with the subjective onset of psychosis rather than treating an unusual belief as a direct chemical readout. It also predicts that dampening salience can reduce the compelling quality of experiences before their interpretation has been revised. This is a theoretical synthesis, not an experiment proving the entire causal chain.^[ref]

The framework makes two separable predictions: inappropriate significance should attach to uninformative cues, and relevant cues may fail to gain appropriate significance. They should be measured independently. In the Salience Attribution Test used by Jonathan P. Roiser and colleagues, visual cues varied along colour and form dimensions, with only one dimension predicting monetary reinforcement. Reaction-time speeding and explicit probability ratings indexed responding to relevant versus irrelevant features. Patients showed reduced adaptive salience; importantly, the diagnostic groups did not differ overall in aberrant salience. Greater explicit aberrant salience occurred in patients with delusions relative to patients without them. The small, predominantly medicated sample, subgroup analysis and multiple comparisons prevent presenting this as a universal schizophrenia deficit.^[ref]

A useful measurement caution follows directly. If equally uninformative features receive noisy ratings $R_1,R_2$, the absolute difference $|R_1-R_2|$ is generally positive even when both estimates are unbiased. For independent normal errors of variance ${\sigma }^2$, its expectation is $2\sigma /\sqrt{\pi }$. A high score can therefore reflect rating noise or poor probability discrimination as well as genuinely assigned significance. Repeated trials, reliability assessment, explicit task comprehension and measurements of adaptive learning are needed to interpret the score.

Biological evidence is compatible with, but does not uniquely identify, the hypothesis. Oliver D. Howes and colleagues used fluorodopa PET to compare people with prodromal symptoms, patients with schizophrenia and controls. Elevated synthesis-related uptake in the associative striatum in the prodromal group links dopamine dysfunction to a stage before established psychotic illness. This was a case-control study; uptake is not a recording of cue-specific phasic dopamine, and a high-risk group does not establish that every individual will develop schizophrenia. It supplies an upstream biological constraint, not a demonstration that a particular neutral cue caused a delusion.^[ref]

Several explanatory steps remain. An account must specify why significance is assigned to this event, how a persistent interpretation is selected, and why contradictory evidence fails to revise it. Attention, memory retrieval, prior beliefs and the social environment can supply those constraints. Dopamine dysfunction need not explain all negative symptoms or cognitive difficulties. Antipsychotic effects cannot alone establish the original cause: altering a downstream component can improve symptoms arising through several pathways. Nor should a medication-associated decrease in ordinary motivation automatically be taken as recovery of selective salience.

A similar perspective is particularly productive for substance use disorder. Incentive salience distinguishes cue-triggered wanting from pleasure. Cindy L. Wyvell and colleagues separately trained a sucrose-seeking response and a reward-predictive cue. Accumbens amphetamine enhanced cue-triggered responding in a test without reward reinforcement, without enhancing sucrose liking measured by taste reactivity. A later sensitization experiment found amplified cue-triggered seeking in a drug-free test after prior exposure. The experiments distinguish motivation from hedonic impact; they do not themselves demonstrate every feature of human addiction.^[ref][ref]

This suggests how drug cues can capture behaviour even when enjoyment has declined or the person intends abstinence. Unlike psychosis, however, these cues were genuinely associated with reward. The shared issue is excessive or poorly regulated motivational control, not necessarily significance attached to statistically irrelevant events.

Anxiety disorders offer a second, more limited analogy: threat-related cues can monopolise attention, avoidance can maintain their significance, and safe cues may be treated as dangerous. In panic disorder, catastrophic interpretation can make ordinary bodily sensations feel threatening. Such accounts concern threat appraisal and discrimination and do not justify assuming identical dopamine abnormalities across diagnoses. Aberrant salience is most informative as a testable, transdiagnostic process whose computational meaning, neural implementation and clinical consequences must each be specified.

Learning and memory provide powerful mechanistic accounts of some psychiatric symptoms, but they are not an adequate single-level definition of all psychiatric disorders. The most useful claim is that abnormal learning, valuation or retrieval can mediate particular symptoms and interact with biological vulnerability and environment. Calling every persistent symptom a “memory” without specifying a computation, mechanism and discriminating prediction would make the account unfalsifiable.

Several causal positions must be separated. A learning abnormality could precede illness and increase vulnerability, mediate the expression of an established biological disturbance, or follow from the illness, medication or adverse experiences. A cross-sectional task deficit does not distinguish these possibilities. Nor does a diagnosis identify one latent parameter: poor performance can result from altered motivation, perception, attention, beliefs or motor output as well as learning. Construct validity and external validity are therefore central, not afterthoughts.

For schizophrenia, an influential proposal links dopamine dysregulation to aberrant salience. Events or internal representations acquire inappropriate significance, and attempts to explain those experiences can contribute to delusions. This creates a possible bridge between neurochemistry and acquired belief. It is a theoretical framework rather than evidence that each psychotic belief is an ordinary reinforced association; psychosis is also not specific to schizophrenia. The account must explain which symptoms it predicts and which remain outside its scope.^[ref]

A pharmacological experiment provides a more direct test of one link. Healthy volunteers performed causal associative learning during randomized ketamine and placebo sessions with functional magnetic resonance imaging. The drug perturbed right-frontal responses related to violated expectations, and individual neural responses were related to subsequently assessed perceptual and belief-like drug effects. This supports an association between altered learning-related processing and psychotomimetic experience. It does not show that ketamine reproduces the developmental course of schizophrenia, or that the observed imaging signal is the unique mediator of a delusion. Pharmacology causally changes the drug condition; the relationship between an imaging response and a symptom remains a separate inferential step.^[ref]

Prospective evidence helps with temporal ordering. Elevated striatal dopamine synthesis capacity was observed before transition to psychotic disorder in a high-risk cohort. That finding is important biological evidence, but a PET synthesis-capacity estimate does not directly measure trial-wise reward prediction errors, and elevated capacity does not establish a specific learning algorithm. It also does not imply that every high-risk person or every diagnosed patient has the same abnormality. A learning account should connect this biological vulnerability to task-defined computations rather than substitute one measure for another.^[ref]

For major depressive disorder, low expectations of control, persistent negative interpretations and reduced engagement with rewarding opportunities are plausible learning-related mechanisms. Learned helplessness provides a specific model of how uncontrollability can change later behaviour; attributional style helps explain generalization and persistence in human accounts. These mechanisms are neither a moral judgment about the individual nor proof that all adverse circumstances are controllable. Sleep, bodily symptoms, neurodevelopmental factors and the diversity of depressive presentations remain part of the explanandum.

Behavioural evidence is more precise than a global claim that depressed people “cannot learn.” In a probabilistic reward task, unmedicated depressed participants showed reduced development of response bias toward a more frequently rewarded stimulus, with analyses implicating integration of reinforcement history rather than total absence of response to an individual reward. This separates one aspect of reward learning from subjective pleasure and from every component of anhedonia. The direction of causation remains open in a cross-sectional comparison.^[ref]

Treatment findings can also link biological and cognitive levels. An acute norepinephrine-reuptake manipulation changed emotional processing in depressed participants before subjective mood changed. That observation is consistent with pharmacologically altered experience processing contributing to later improvement; it does not prove that the early task change mediates every later clinical response. Learning and pharmacology can interact rather than constitute mutually exclusive treatments. A drug might alter the conditions under which corrective experiences are encoded, while behavioural intervention supplies those experiences.^[ref]

The learning perspective generates concrete treatment hypotheses. In threat disorders, extinction learning suggests improving safety learning and its retrieval across contexts, rather than merely suppressing an immediate response. A randomized pilot study combined D-cycloserine or placebo with virtual-reality exposure to heights in people with acrophobia. Greater improvement in the adjunct condition provided an early proof of concept for facilitating a specified learning experience. The small, specific trial does not establish a general cognitive enhancer for all diagnoses, and an intervention that facilitates learning need not selectively strengthen desirable information. Treatment timing and the quality of the learning episode are therefore empirical issues.^[ref]

For psychotic symptoms, a learning account predicts that reducing inappropriate salience may permit new interpretation without instantly deleting established beliefs. For depression, it motivates tests of changes in reward sensitivity, information integration and expectations, with functional recovery as an outcome. These are treatment-development hypotheses, not conclusions that a single task parameter should dictate care. Symptom improvement, correction of a proposed mechanism and prevention of relapse require separate measurements; changing a laboratory score is not sufficient evidence of benefit.

The broad claim becomes convincing only when models predict specific patterns, survive controls for performance confounds, and show temporally appropriate changes linked to symptoms. Comparisons with unaffected high-risk individuals, longitudinal designs and mechanism-sensitive randomized trials can distinguish vulnerability, mediation and consequence. Learning and memory are therefore valuable explanatory levels and possible treatment targets, especially for particular acquired responses and beliefs. They do not replace the biological, developmental and social causes of psychiatric illness, and they should not turn diagnostic heterogeneity into one assumed defect.

Reward prediction error remains a strong account of particular phasic dopamine signals and their contribution to learning; it is inadequate as a universal description of every dopamine neuron, projection and timescale. The question is best answered by separating the computational hypothesis, measurements of naturally occurring activity, and evidence that manipulating an error-like signal changes later learning.

In temporal-difference learning, a prediction is updated usingwhere $V(s)$ is the value function for state $s$, $r_{t+1}$ is the next reward, $0\le \gamma \le 1$ discounts later return, and $0<\alpha \le 1$ is the update step size. The terminal value is zero. A reward prediction error is therefore not simply reward, pleasure or salience. Its sign depends on what was expected and when. An unexpected reward can generate a positive error; an accurately predicted reward can generate little error; omission of an expected reward can generate a negative error. A cue can produce an error when it changes the prediction of future return even though no immediate reward is delivered.

Primate recordings motivated this account because phasic responses changed with learning and reward expectation rather than responding identically to every reward. Linking these patterns to a quantitative learning rule provided predictions about cue responses and outcome omissions that a generic “reward neuron” label lacked. However, extracellular identification and population sampling matter, and the computational state representation is an assumption. A model fitted with a poor description of what the animal knows can misclassify a genuine expectation-related response as a theoretical failure.^[ref]

Association-learning experiments make the interpretation more than a curve-fitting exercise. In blocking, an established cue already predicts the reward, reducing learning about an added cue. Reward-timed optogenetic activation of dopamine neurons could overcome this blocking, supplying an error-like teaching event despite the normally expected reward. The relevant endpoint was subsequent learning about a cue, not just movement during stimulation. This supports a causal teaching contribution but does not establish that artificially synchronized activation reproduces every feature of an endogenous code.^[ref]

The negative side of the error also has intervention evidence. Brief inhibition of targeted dopamine neurons around reward processing produced later learning changes resembling those expected from a negative error, with behavioural controls addressing distraction or immediate aversion. This strengthens a bidirectional teaching account beyond sufficiency of excitation alone. The imposed pause still tests a particular population and task, and inhibition can affect processes other than the proposed scalar signal. Appropriate timing controls and later unstimulated testing are essential to the interpretation.^[ref]

Population heterogeneity can refine the hypothesis rather than simply refute it. Distributional reinforcement-learning work proposed multiple channels differing in optimism and in their relative sensitivity to positive and negative errors. Mouse single-unit data showed systematic variation in response reversal points consistent with this account. One reward can then be better than expected for one channel and worse for another. The finding challenges a single shared scalar prediction broadcast identically by all cells, while retaining prediction-error-like computations. It does not make heterogeneity alone proof of a particular distributional algorithm.^[ref]

Other heterogeneity is not readily summarized as reward error. Cell-resolved imaging during a mouse virtual-navigation task found dopamine-neuron signals related to sensory, motor and cognitive variables, with structured differences across neurons. Regression and task design can separate some predictors, but correlated movements and expectations limit unique interpretation. Such findings make a universal scalar account implausible; they do not eliminate the well-controlled reward-error results in other populations. Nor should every movement-related signal be re-labelled as error without an independently specified model that predicts it.^[ref]

The measurement compartment is another limitation. A comparison of identified VTA cell spiking with nucleus-accumbens dopamine release in a common decision task found release dynamics related to evolving reward expectations without corresponding changes in recorded cell-body spiking. Local release regulation therefore matters: a terminal sensor and a somatic spike train need not report the same quantity. This is especially important for interpreting slower ramps or motivational effects as direct readings of a broadcast phasic error.^[ref]

Further qualifications concern learning itself. An error depends on subjective reward, inferred state, uncertainty and task structure; it is not calculated from the experimenter’s reward schedule alone. Values informed by a model of the environment can also contribute, so identifying an error-like signal does not automatically prove exclusively model-free learning. Conversely, a very flexible state model can accommodate almost any observation unless its representation and parameters are constrained independently. Good tests manipulate expectation while matching outcome and movement, and compare explicit alternatives on held-out behaviour and neural data.

The account is thus still appropriate when stated at the right resolution: some temporally precise dopamine activity supplies expectation-dependent teaching signals, supported by both recordings and causal learning manipulations. A complete account of the dopamine system must additionally explain projection-specific coding, local transmitter dynamics, motivation and movement. The productive revision is a family of constrained computational and circuit hypotheses, rather than either one molecule–one number or abandonment of prediction error altogether.

The SOP model family specifies testable principles of stimulus processing and association, but it does not yet have a uniquely established neural implementation. Its behavioural breadth should be distinguished from evidence identifying the biological counterparts of its states, transition rules and separate response systems. The strongest neurobiology concerns processes that the models require—priming, associative retrieval and differentiated response pathways—rather than a one-to-one mapping of every symbol onto a brain structure.

Ivan Pavlov’s classical conditioning tradition made stimulus history and the measured conditioned response objects of controlled experiment. The SOP model introduced by Allan R. Wagner represents stimuli as elements distributed among inactivity $I$, primary activation $A_1$ and secondary activation $A_2$. External presentation promotes inactive elements into $A_1$; activity then passes through $A_2$ before returning to inactivity. A learned associate can activate an expected event directly into $A_2$. Recently or associatively activated elements are less available for renewed primary processing, distinguishing self-generated priming from retrieval-generated priming. A contemporary quantitative analysis shows how these dynamics can generate several habituation regularities; these are simulations, not measurements of neuronal states.^[ref]

The association rules make the account more specific than a generic fading trace. In the original formulation, concurrent $A_{1,CS}$ and $A_{1,US}$ activity promotes excitation, whereas $A_{1,CS}$ with $A_{2,US}$ promotes inhibition. The Modified SOP model additionally permits learning involving a cue activated into $A_2$: matched-state combinations can strengthen excitation and mismatched-state combinations inhibition. This allows an absent cue’s association to change through retrieval, addressing retrospective revaluation. The model is therefore not simply equivalent to a reward-error equation or to synaptic fatigue.

Human causal-judgment experiments motivated an important constraint. Participants learned relations among compound cues and an outcome, then received information about a companion cue without the target. Retrospective changes in judgments depended on the consistency of earlier within-compound pairings. This supports a role for learned cue–cue relations in revaluation, but the behavioural result does not identify synapses implementing the modified state-pair rules. Other retrieval or inference models can also use knowledge of those relations.^[ref]

Neural adaptation alone is insufficient to explain retrieval-generated effects. A transient reduction in response to a repeated physical stimulus could result from receptor adaptation, local synaptic dynamics or wider circuit changes, whereas context-specific long-term priming requires an account of the context’s learned relation to the event. A biological test must distinguish an unfamiliar stimulus from an unexpected combination of familiar stimuli and control attention, orienting and stimulus exposure. Reduced firing cannot simply be labelled $A_2$: the theory permits secondary activity to be behaviourally silent or have a response-system-dependent effect.

Hippocampal lesion experiments provide evidence at that process level. Rats learned audiovisual sequences in which particular sounds preceded particular lights. Switching the familiar sound–light combinations renewed visual orienting in controls but not in hippocampal-lesioned animals, despite relatively preserved habituation to visual stimuli and evidence of sequence associations. This implicates the hippocampus in associative mismatch rather than novelty alone. It is compatible with a contribution to expectation or retrieval, but does not uniquely show that the hippocampus is $A_2$, nor that every SOP model process is hippocampal.^[ref]

Receptor perturbation further separates short- and long-term components. Mice lacking the GluA1 AMPA-receptor subunit showed impaired short-term spatial novelty performance but enhanced long-term performance under the tested interval conditions. Manipulating exposure and test intervals helped distinguish rapid recency-dependent effects from longer-lasting learning. This is consistent with separable processes of the kind emphasized by the SOP model, rather than an undifferentiated stronger-versus-weaker memory. Nevertheless, a lifelong mutation can alter development and network function, and task dependence prevents identifying GluA1 with a single abstract transition parameter.^[ref]

Associative access to an absent outcome has additional circuit support. In rat sensory preconditioning, orbitofrontal inactivation impaired behaviour guided by an indirectly inferred outcome while sparing behaviour supported by the directly reward-paired cue. The manipulation also altered learning when inferred value normally acted as a blocker. This shows that retrieved associative structure can influence both performance and new learning. It is relevant to the demands of Modified SOP model, but does not demonstrate that the neural representation is in exactly $A_2$ or that its plasticity has the model’s stipulated sign.^[ref]

The Affective extension of SOP, developed by Allan R. Wagner and Susan E. Brandon, distinguishes sensory-specific and emotive components with different processing time courses and response consequences. Behavioural experiments in rabbits showed modulation of a discrete conditioned eyeblink by another putatively emotive conditioned stimulus. Such dissociations motivate separate response components. They do not locate a complete emotive representation in one nucleus. Basic conditioned-eyelid experiments independently implicated cerebellar nuclear circuitry through lesions, recordings and stimulation, illustrating why a motor conditioned response cannot simply be assigned to the same mechanism as every autonomic or emotional effect.^[ref][ref]

The remaining challenge is discriminative implementation. A candidate neural model should specify what an element is, how direct and retrieved activity differ, how relevant overlap controls plasticity, and how sensory and affective components affect separate outputs. Manipulating receptor function or recording reactivation is not enough unless alternative mechanisms make different predictions. Neuromodulators could regulate transition rates, representation selection or plasticity, but there is no justified general identity such as “dopamine is $A_1$” or “serotonin is $A_2$.” Multiple biological mechanisms could realize the same functional state description.

The SOP model family is therefore a valuable framework connecting stimulus history, expectation and response organization. Primary lesion, receptor and inactivation evidence supports several component processes. Its comprehensive behavioural ambitions exceed what is currently established about its precise neurobiology; the next useful experiments distinguish direct activation from retrieved activation and test the associated plasticity rules, rather than merely find a neural correlate of one familiar behavioural effect.

Correlative and causal approaches answer different questions, and neither is dispensable for understanding neuromodulation. An intervention is generally stronger evidence about the consequence of changing a specified target, while observation is essential for discovering the naturally occurring signal, its heterogeneity and the contexts in which it appears. Usefulness depends on the claim, measurement and assumptions, not simply on whether an experiment uses an optogenetic actuator.

A correlation between a neuromodulatory signal and behaviour can arise because the signal influences the behaviour, behaviour changes the signal, or a third variable influences both. Reward expectation, movement and arousal often covary. Temporal precedence helps reject some accounts but does not eliminate an unmeasured common cause. Detailed task manipulations can nevertheless make correlative evidence much more discriminating: varying expectation while holding delivered outcome constant tests a different hypothesis from merely comparing rewarded and unrewarded trials.

The dopamine prediction-error account illustrates this value. Patterns across unexpected rewards, predictive cues and omitted expected rewards constrained an explicit computational hypothesis. A generic rise in activity at a reward would not distinguish reward receipt, expectation and error. Recording before intervening helped identify the timing and task conditions in which a putative teaching signal should be manipulated. Without that information, a stimulation experiment might show that a pathway can change behaviour while leaving the proposed natural computation largely unspecified.

Observations also reveal what population averages hide. Cell-resolved dopamine imaging showed structured sensory, motor and cognitive heterogeneity during a navigation task. Comparing identified dopamine spiking with accumbens release revealed that evolving release dynamics need not mirror recorded somatic activity. These findings constrain the target and measurement level of subsequent interventions. They are not failures of observation: a perturbation of an entire heterogeneous population could obscure precisely the functional distinctions that recording discovers.^[ref][ref]

Interventions can then test a well-specified causal prediction. Reward-timed activation of dopamine neurons overcame associative blocking, changing subsequent cue learning; brief inhibition supported a negative-error-like learning effect. These manipulations are stronger evidence for a teaching contribution than a correlation alone. Yet the manipulated population and timing must match the proposed endogenous signal, and later unstimulated testing is needed to separate acquired learning from immediate performance effects. Sufficiency to impose an effect is not proof that the artificial activity pattern occurs naturally or is the only possible teaching route.^[ref][ref]

Claims of functional necessity and sufficiency are conditional. Selective inhibition that impairs performance can support necessity under the tested conditions, but may act through sensation, movement or motivation rather than the hypothesized computation. A null effect can reflect redundant circuits, incomplete target engagement or compensation. Conversely, stimulation can recruit an unnatural synchrony, reverse-propagating activity or multiple collateral targets. Changing neurons identified by one transmitter can also change co-transmission, so the intervention is not automatically selective for that chemical signal at one synapse.

Pharmacological perturbations make similar assumptions visible. A systemic drug affects receptors according to exposure and selectivity, not according to the experimenter’s functional label. Time of testing, peripheral actions, active metabolites and off-target drug effects matter. The chemogenetic ligand clozapine-N-oxide provides a concrete example: receptor-binding and distribution work showed that conversion to clozapine can account for central DREADD activation. Thus an apparent designer-receptor intervention cannot be assumed to have the intended pharmacological isolation merely because the receptor was genetically targeted. Ligand-only controls and exposure measurements improve the causal interpretation.^[ref]

Measurement is therefore required inside causal experiments as well as before them. Histology can confirm targeting but not the actual moment-to-moment physiological effect; electrophysiology or transmitter measurements can test target engagement. Calcium signals, fibre photometry, voltammetry and spikes have different spatial, temporal and chemical limitations. Converging methods are more persuasive when they agree at their common resolution, while disagreements can identify local regulation or a mistaken measurement model rather than simply experimental failure.

Causal mediation is a further step. Randomizing a drug can establish an effect of the assigned treatment, but correlating a post-treatment neural signal with symptom change does not prove that signal mediated the effect. Both may reflect another drug consequence. Likewise, statistically controlling for movement can remove a confound in one causal model but remove a genuine intermediate mechanism in another. A causal diagram and an explicit target estimand should guide adjustment rather than a rule to regress out every variable associated with behaviour.

Human and animal approaches have complementary scope. Fine cellular interventions can provide internal validity in a restricted animal task, while human longitudinal observations can test whether candidate abnormalities precede illness and whether the same behavioural dimensions matter clinically. Ethical and practical constraints preclude arbitrary human lesions or perfectly selective transmitter manipulation. It would be unhelpful to discard clinically relevant observation for failing to perform an impossible experiment; equally, a prospective association should not be relabelled as a proven biological cause.

The most productive programme iterates among formal models, naturalistic measurements and selective perturbations. Specify the computation, identify its predicted signal, vary task factors that distinguish alternatives, and intervene with appropriate timing, targeting and performance controls. Re-test the resulting behavioural and neural predictions across contexts. Causal experiments generally have an advantage for a precise intervention question, but correlative approaches remain essential for deciding what should be changed and what the causal result actually explains.

A midbrain region containing dopamine neurons and other cell types. Identifying the region does not establish the transmitter identity of every recorded unit.