Neonatal imitation 2026-10-25
Model-specific action matching during the neonatal period. Early positive reports and subsequent disputed replication and reanalysis make this a useful case for distinguishing whether an action occurs, whether it is selectively elicited by its matching model, and whether it supports a broad theory of imitation.[ref]
Imitation of facial and manual gestures by human neonates by Andrew N. Meltzoff and M. Keith Moore. Science 198, 75–78; modelled facial/manual actions and coded infant responses
Greater transparency and reproducibility have changed developmental psychology's estimates, measurement practices and theoretical confidence, but have not simply replaced a collection of “true” findings with “false” ones. Computational reproducibility asks whether the same data and documented analysis yield the same result. Scientific replication asks what happens with new observations. Preregistration makes planned tests distinguishable from exploratory choices; open materials permit checking whether ostensibly similar studies actually use the same procedure. None of these, by itself, guarantees construct validity or representative sampling.
The marshmallow task illustrates a substantive change in interpretation. Walter Mischel's early experiments offered preschoolers a smaller immediate reward or a preferred delayed reward and manipulated how attention was directed during waiting. Distraction increased delay. The original experimental programme therefore already showed sensitivity to cognitive strategy and setting, rather than a fixed quantity of virtue revealed by a treat. Later predictive associations are a different claim from those immediate experimental effects.[ref]
Cognitive and attentional mechanisms in delay of gratification by Walter Mischel and others. Journal of Personality and Social Psychology 21, 204–218; experimental manipulations of attention during waiting
Tyler W. Watts and colleagues revisited long-term prediction in a larger and more diverse cohort. Their 2018 conceptual replication used a waiting task at fifty-four months and adolescent achievement outcomes. Associations were substantially reduced after adjustment for family background, early cognitive abilities and the home environment; much of the achievement difference was concentrated at the lowest waiting times. An important measurement difference was the seven-minute ceiling, rather than treating all waiting tasks as interchangeable. These results weaken a simple causal story in which training a preschooler to wait longer should, by itself, transform later attainment. They do not show that self-regulation never matters, because adjustment can remove both confounding and potentially meaningful pathways.[ref]
Revisiting the marshmallow test by Tyler W. Watts and others. Psychological Science 29, 1159–1177; Delay of gratification measure, adjusted models and measurement limitations
Celeste Kidd and colleagues supplied experimental evidence for one alternative mechanism: environmental reliability. Before the waiting task, children encountered an adult who either fulfilled or failed promises about appealing materials. Children subsequently waited longer after reliable experiences. The observation matters because apparent “poor self-control” can be an understandable response to expecting that a promised reward will not arrive. This small experiment supports contextual causation under its own conditions, not a claim that all socioeconomic differences in waiting are caused by trust. Together, the studies shift the target from an isolated child trait to the interaction between control, expectations and circumstances.[ref]
Rational snacking by Celeste Kidd and others. Cognition 126, 109–114; reliable versus unreliable experimenter manipulation
Neonatal imitation illustrates why replicability also depends on the inferential contrast. Andrew N. Meltzoff and M. Keith Moore's 1977 report described very young infants matching facial and manual gestures. In this paradigm an adult models an action and infant responses are recorded and coded; the crucial question is whether the infant produces a given action preferentially after its matching model, not merely whether the action occurs. An increase in tongue protrusion after interesting stimulation is insufficient unless appropriate nonmatching controls exclude general arousal or exploration.[ref]
Imitation of facial and manual gestures by human neonates by Andrew N. Meltzoff and M. Keith Moore. Science 198, 75–78; modelled facial/manual actions and coded infant responses
Janine Oostenbroek and colleagues tested 106 infants longitudinally with nine social and two nonsocial models at one, three, six and nine weeks. Their broad comparisons did not support model-specific imitation, although restricted cross-sections could resemble earlier positive findings. This challenged a general innate-imitation account and highlighted the consequences of selecting particular gestures, ages and contrasts after observing results. Yet the study's breadth also created substantive questions about fatigue, presentation and how each action was scored.[ref]
Comprehensive longitudinal study challenges the existence of neonatal imitation in humans by Janine Oostenbroek and others. Current Biology 26, 1334–1338; nine social and two nonsocial models across four ages
The subsequent requested-data reanalysis by Meltzoff and colleagues reported tongue-protrusion matching under pooled-control contrasts and criticised features of the elicitation and scoring procedures. That was not an independent replication: it reused the same observations. It nevertheless shows the scientific value of access to raw data. The disagreement became one about estimands, comparison sets and measurement sensitivity that another preregistered experiment could adjudicate, rather than an unsupported declaration that one team must be wrong. Evidence for one gesture would also not automatically vindicate a broad, domain-specific imitation mechanism.[ref]
Re-examination of Oostenbroek et al. (2016): evidence for neonatal imitation of tongue protrusion by Andrew N. Meltzoff and others. Developmental Science 21, e12609; requested raw data, pooled-control tongue-protrusion analyses and design criticisms
ManyBabies 1 provides a constructive counterexample to the notion that open developmental research only dismantles effects. In a Registered report, an international collaboration used common speech stimuli and prespecified analyses to test infant-directed versus adult-directed speech preferences. The final sample included 2,329 infants from 67 laboratories. A preference for infant-directed speech survived, while its magnitude varied with age, language experience and measurement method. Laboratories used central fixation, head-turn preference or eye tracking procedures, making methodological variability an object of analysis. The result supports a reproducible average preference without treating the same effect size as universal for every infant or cultural setting.[ref]
Quantifying sources of variability in infancy research using the infant-directed-speech preference by The ManyBabies Consortium. Advances in Methods and Practices in Psychological Science 3, 24–52; Participants, preregistered analyses and Discussion
Several changes to our understanding follow. First, an established group-level tendency can coexist with weak prediction for an individual child. Secondly, stimulus choice, stopping rules, exclusions and coding decisions can change what has actually been measured. Thirdly, heterogeneous outcomes can identify theoretically useful boundary conditions, but a moderator discovered after a failure needs a new test rather than serving as an unlimited excuse. Finally, openly available null results reduce selective visibility of successful studies, improving the evidential base used to build developmental accounts.
The causal claim about reform itself should remain modest. Larger samples, broader recruitment and better controls often accompany transparency, so a change in an estimate cannot be attributed uniquely to preregistration. A perfectly reproducible analysis can reproduce a biased model, and an underpowered replication may be inconclusive. Reliability and measurement equivalence are especially important for looking-time and single-trial child tasks; independent coding and predefined exclusion rules help, but do not establish the intended psychological construct. Open child data also require appropriate consent and controlled access where identification is possible; useful transparency need not mean publishing recognisable videos.
By 2022, these developments justified a more conditional developmental science: waiting reflects more than a decontextualised trait; claims of newborn action matching require precise controls; and an infant speech preference can be robust while method-sensitive. The main advance is a tighter connection between a theoretical claim, an explicitly defined measure and evidence that others can inspect or independently test. Transparency changes what can be learned from disagreement, while replication establishes how far a finding travels.
Make the development process inspectable and separate exploratory decisions from independent confirmation. Create a preregistration for the validation stage before examining its outcome-relevant data, specifying recruitment, stopping rules, exclusions, missing-data treatment, item scoring, estimator, factor structure and criteria for evaluating it. The exploratory stage can also have a recorded protocol, including a planned parallel analysis rule; label data-driven item revisions and model changes as exploratory rather than disguising them as prior predictions. Document deviations and their reasons.[ref]
Preregistration by Center for Open Science. What is Preregistration?; planned versus unplanned work; FAQ on detail and prior data knowledge
Keep an immutable copy of the collected data, record cleaning steps, and provide a data dictionary, final item wording, scoring instructions and executable analysis code. Record software and package versions, random seeds for simulations or bootstrap calculations, and the steps needed to regenerate tables from the permitted data. These support computational reproducibility; an independently collected validation study or scientific replication addresses a different question.
Share appropriately de-identified data where consent and re-identification risk permit, with controlled access or a clear restriction statement where they do not. Protect lecturer and rater identities and disclose the pilot's convenience sampling and all item-selection steps. Report inconclusive and unfavorable findings as well as favorable ones. A registered report can obtain peer review of the validation plan before its results are known, reducing outcome-dependent publication incentives.[ref]
Registered Reports by Center for Open Science. Details and Workflow: Stage 1 review, in-principle acceptance, Stage 2 reporting
A useful response to the demarcation problem should identify practices which make claims answerable to evidence without equating scientific status with either truth or prestige. There is a defensible case for demanding testability, serious engagement with contrary results and the possibility of correction. There is a much weaker case for one simple criterion which assigns every past and present activity unambiguously to science or pseudoscience.
Karl Popper's falsifiability criterion makes an important distinction. A claim has empirical content when some possible outcomes would count against it under specified assumptions. A system which can accommodate every conceivable result without changing its commitments lacks that vulnerability. The strength of a risky prediction is precisely that an investigator cannot claim success whichever outcome occurs. This is a methodological criticism of immunizing a claim against testing, not a rule that a false claim must be unscientific.[ref]
Science: Conjectures and Refutations by Karl Popper. Chapter 1: risky predictions, falsifiability and conventionalist strategies for avoiding refutation
A simple counterexample shows why falsifiability alone cannot settle the whole question. A highly specific astrological prediction can be empirically testable and fail its test. Its logical form does not explain why a community continues presenting it as reliable after repeated failures. Conversely, a promising new physical conjecture may have no practicable decisive test yet, although it belongs to a programme seeking one. Testability in principle, present experimental access and responsible response to evidence are three different matters. Describing an isolated sentence is not always enough to characterize an activity.
Nor does a failed prediction logically identify its culprit. If a theory and an auxiliary hypothesis imply an outcome , a contrary outcome establishes , not uniquely . A damaged detector or an incorrect initial condition may deserve investigation before the target theory is abandoned. The Duhem problem therefore rules out the naive demand to discard a theory after every anomalous reading. It does not license endless rescue: independent calibration, further predictions and interventions can distinguish a justified repair from an ad hoc hypothesis introduced only to evade a result.[ref]
The Aim and Structure of Physical Theory by Pierre Duhem. Part II, chapter VI, section 2: experimental testing concerns a group of theoretical assumptions
Imre Lakatos accordingly evaluates a scientific research programme through its development. Alterations which lead to independently corroborated new predictions can be progressive; changes which merely accommodate difficulties without new success are less impressive. The relevant contrast concerns a pattern of inquiry over time. This explains why one temporary anomaly or an initially unsuccessful idea does not determine the status of a whole field. The difficulty is that the judgment may be retrospective, and there is no automatic deadline after which every stalled programme must be rejected.[ref]
Science and Pseudoscience by Imre Lakatos. Discussion of research programmes, novel facts and progressive versus degenerating changes
A defensible assessment therefore combines several considerations: specificity of claims, quality and independence of tests, handling of uncertainty, disclosure of methods, willingness to revise, and success in producing further research. None is individually sufficient. Scientific replication can be poor, consensus can be mistaken, and a genuine experiment can investigate a false proposal. These qualifications do not make all approaches equally credible; they make the reasons for a classification more informative than a label alone.
The resulting criterion is satisfactory as a fallible standard for evaluating inquiry, rather than as an exceptionless definition discovered by inspecting the grammar of a claim. For a practical decision, it is often more precise to identify a failure, such as an uncontrolled comparison or a refusal to specify what would count against a prediction. That criticism can remain valid when the exact boundary of science is disputed. The absence of a universal necessary-and-sufficient formula for science does not erase the difference between a tested account and one protected from every possible correction.
Theory-ladenness can affect the concepts used to describe an observation, the interpretation of an instrument and even which features attract attention. A detector's pulse height is not already a particle's energy: the inference depends on measurement calibration, response assumptions and background subtraction. This defeats a particularly strong empiricism according to which every scientific claim rests on a completely theory-free observational foundation.
It does not follow that observation can never constrain theory. The crucial distinction is between relying on some background account and presupposing the truth of the particular target under test. One can calibrate a detector with independently characterized sources and compare measurements from detectors with different systematic errors. Agreement across these routes is more compelling than repeated use of one disputed reduction procedure. Scientific replication and tests with deliberately varied apparatus can expose shared assumptions rather than merely repeat them.
An empiricist can therefore adopt a fallible, public evidential practice. The observable phenomenon need not be linguistically uninterpreted to have an evidential role. Reports can be translated sufficiently for disagreement, and raw records can be retained so that rival analyses can be compared. Where all purported evidence is defined through the target theory itself, the circularity is a real problem; the reply must supply independent checks, not simply assert objectivity.
Constructive empiricism offers one further distinction: accepting a theory for its empirical adequacy does not require believing its full story about every unobservable entity. Its proponent can use a theoretical account to organize instruments without identifying use with unconditional belief.[ref]
Constructive Empiricism Now by Bas van Fraassen. sections II–III: instrumental detection and acceptance versus belief
Theory-ladenness therefore undermines an infallible foundation, while leaving room for constrained, corrigible empiricism.Scientists should take sceptical challenges seriously when they reveal unsupported assumptions or distinguishable sources of error. They need not obtain a proof against every conceivable deception before conducting an experiment. This position concedes the force of some philosophical scepticism while rejecting certainty as the normal standard of scientific warrant.
There are several importantly different challenges. Global philosophical scepticism questions whether an external world exists at all. The problem of induction questions the justification for extending observed regularities to unobserved cases. Local criticism questions a particular instrument, sample or inference. A response appropriate to the last challenge need not answer the first. Repeating a measurement with a second instrument can expose a faulty sensor; it does not prove that all experience is independent of a deceiving agency.
David Hume's challenge to inductive reasoning is especially relevant. A finite record does not deductively entail a universal generalization. Nor can the inference be justified simply by saying that past uses of inductive reasoning succeeded: that argument itself extends a past regularity beyond the observed cases. The possibility that an observed regularity fails tomorrow is not a contradiction. This identifies a real gap between deduction and empirical projection.[ref]
An Enquiry Concerning Human Understanding by David Hume. section IV, parts I–II, especially paragraphs 16–21 on the circular appeal to past experience
For example, suppose a metal's measured resistance agrees with throughout a tested temperature interval. Both that expression and a rival with an extra term zero throughout the interval fit those measurements exactly, yet can disagree outside it. Additional measurements can discriminate specified rivals, but no finite collection excludes every possible future deviation. The scientific lesson is to state the tested domain, physical assumptions and uncertainty. A linear electrical resistance model can be useful without being a universal law; a phase transition is an independently intelligible reason why extrapolation might fail.
One might reply that successful prediction and scientific replication justify confidence. They certainly distinguish a tested model from an arbitrary guess under the ordinary standards of inquiry. They do not supply the noncircular proof demanded by radical philosophical scepticism. A Bayesian inference similarly makes the dependence on prior probabilities and likelihoods explicit; it does not deduce the correctness of those commitments from no premises. Acknowledging this limit is preferable to claiming that a mathematical updating rule dissolves the problem of induction.
The sceptic nevertheless owes us an account of the practical consequence. If two global possibilities give exactly the same observable consequences under every available intervention, their difference does not identify which laboratory procedure to change. A scientist can suspend judgement about that difference while comparing models that make different predictions. This is a defence of proceeding under fallible standards, rather than a refutation of global philosophical scepticism.
By contrast, local criticism can reorganize an entire research programme. If an apparent signal might be instrument drift, useful responses include independent calibration, reversing a controllable input, measuring an appropriate control and conducting scientific replication. Those procedures make the proposed error account answerable to evidence. Merely imagining that every control could also be deceptive adds no further discriminating test.
The defensible policy is therefore fallibilism: confidence proportionate to the testing and domain, openness to identifiable error, and no claim to absolute foundations. Sceptical arguments matter most when they improve the questions asked of a model. Their inability to be conclusively defeated does not make every model equally warranted or make scientific work irrational.
A defensible demarcation evaluates a practice's accountability to evidence, rather than treating one syntactic feature of a claim as a complete test. There are clear cases of pseudoscience, even if there is no exceptionless rule that classifies every immature or disputed research programme immediately.
Karl Popper's falsifiability criterion captures an essential insight: a claim has empirical content when some possible outcomes conflict with it. A prediction that a specified sample will always have the same electric charge would exclude outcomes that a claim allowing every possible result does not. If apparent failures can always be redescribed as successes, accumulated favourable stories do not supply a demanding test. Popper also emphasizes risky predictions and warns that protective modifications can destroy a theory's testability.[ref]
Science: Conjectures and Refutations by Karl Popper. chapter 1, section I, numbered conclusions on risky prediction, refutability and conventionalist rescue
However, falsifiability alone is not sufficient. A bare numerical prediction chosen without evidence may be perfectly falsifiable. So can a particular astrological claim. Testing such a claim seriously can be scientific work; a movement that advertises the claim while systematically ignoring its failure can still be pseudoscience. Logical vulnerability and actual willingness to learn from tests are different properties. Moreover, a false theory can be scientific, while a true guess can lack a scientific basis. Demarcation is not identical to separating true from false statements.
Nor should every discrepancy lead to immediate rejection. Predictions depend on instruments, initial conditions and auxiliary hypotheses. A detector fault may explain a failed prediction more reasonably than rejection of a well-tested physical law. Imre Lakatos therefore evaluates sequences of theories and their treatment of anomalies: a progressive programme generates new testable successes, whereas a degenerating one primarily adjusts to what is already known.[ref]
Science and Pseudoscience by Imre Lakatos. transcript discussion of Newtonian anomalies and the contrast between progressive and degenerating research programmes
This improves on assessing an isolated sentence, although deciding when a programme has had enough time is itself contestable.A workable standard combines several mutually supporting demands. Claims should specify relevant evidence and conditions of application; tests should expose them to substantial risk of failure; data and methods should permit criticism and appropriate scientific replication; and revisions should improve independently assessable performance. The crucial unit is a sustained practice presenting itself as scientific knowledge. Persistent selective reporting, explanations invented solely to absorb each failure, and refusal to permit competent scrutiny undermine that claim to authority.
For a concrete illustration, suppose a proposed material emits a new kind of signal. Initially poor understanding does not make the proposal pseudoscience. Researchers could specify detector settings, compare blinded signal-present and signal-absent trials, publish failures, and seek independent scientific replication. Contrast a practice that discloses only positive trials, changes the claimed conditions after each negative result, and gives no independent test of the revised conditions. The difference concerns the treatment of evidence, even if both practices use technical language and the same apparatus. The comparison is hypothetical; it does not depend on attributing misconduct to a named group.
This standard has borderline cases. A promising theory can temporarily lack accessible tests, and a programme can rationally survive a period of failure. It should then be represented with the corresponding uncertainty, rather than sold as established. Longstanding prestige, popularity and institutional affiliation are evidence about a community at most; they cannot replace examination of its methods.
Such a criterion is fallible and allows degrees, but that is compatible with useful discrimination. The best boundary identifies practices that systematically evade the evidential responsibilities they claim to meet. It preserves Popper's demand for genuine exposure to failure while incorporating the historical fact that responsible science tests and revises interconnected systems of claims.
The history of science can constrain philosophical claims and support conditional methodological lessons. It cannot, without further premises, turn a record of what scientists did into a rule about what everyone ought to believe. Its strongest contribution is often to reveal that an apparently obvious account of science was too simple.
Consider the claim that scientific progress always adds new facts while preserving every central accepted proposition. The transition from Newtonian mechanics to special relativity supplies a counterexample. Under a Galilean transformation, temporal separation is absolute; a Lorentz transformation mixes time and space coordinates. These are incompatible accounts at unrestricted scope. Yet Newtonian mechanics retains great predictive value at low velocity. Historical change therefore supports a distinction between wholesale accumulation of truths and more limited retention of successful relations. It neither shows that all past knowledge was worthless nor guarantees that current fundamental theories are final.
Thomas Kuhn makes the replacement of central commitments, problems and standards central to his account of scientific revolutions. His introductory examples concern changes that reorganize inquiry, rather than merely adding one item to a fixed inventory.[ref]
The Structure of Scientific Revolutions by Thomas Kuhn. Introduction: revolutionary changes in commitments, accepted problems and scientific practice
This supports a warning against reading present categories unaltered into the past. It does not establish that different scientific communities can never compare evidence or understand one another. Semantic or methodological change is weaker than the claim that rational comparison is impossible.A second lesson concerns anomalies. If actual successful inquiry sometimes retains a theory while examining instruments or auxiliary hypotheses, a methodology demanding abandonment at the first discrepancy misdescribes that inquiry. This matters because experimental predictions concern a package, not a lone sentence. Lakatos's comparison of progressive and degenerating programmes turns attention to whether retaining a framework produces independently corroborated developments.[ref]
Science and Pseudoscience by Imre Lakatos. transcript: scientific theories' tenacity and the need to distinguish successful development from retrospective adjustment
The lesson is conditional: persistence is defensible when it opens productive tests and alternatives, not simply because famous scientists once persisted.The history of science also puts pressure on scientific realism. If empirically successful theories have posited entities later rejected, success alone cannot guarantee literal truth. Larry Laudan's historical challenge targets a proposed explanatory connection between success, reference and approximate truth.[ref]
A Confutation of Convergent Realism by Larry Laudan. section 3, historical examples used to challenge success as a guarantee of reference
But a list of failures cannot establish the fraction of successful theories that fail unless the population and selection process are specified. Cases chosen precisely because they were overturned are an especially poor basis for an unqualified numerical forecast about present theories.There is also a distinction between a historical cause of belief and a justification for belief. Training, instrumentation and institutions affect which theories scientists can develop and test. This makes science a human activity without showing that its results are merely expressions of local preference. A historically informed evaluation should compare the evidence available at the time, the competing proposals, and the costs of different mistakes. Otherwise, hindsight converts the eventual winner's assumptions into premises that earlier participants supposedly should already have known.
These conclusions are philosophically important because they change how we allocate trust. They favour confidence differentiated by claim and domain, explicit attention to auxiliary hypotheses, and humility about extrapolation. They also favour institutions that preserve criticism and scientific replication, because individual ingenuity does not ensure reliable correction. The further normative premise is explicit: methods deserve support insofar as they help inquiry detect error and achieve its evidential aims.
History consequently offers counterexamples to universal philosophical generalizations and evidence for competing explanations of scientific change. It does not choose a unique epistemology automatically. Its value lies in making accounts of rationality and progress answerable to what careful investigation actually reveals, while keeping the passage from description to evaluation open to argument.
Software defects can make a scientific result wrong even when another researcher reproduces it exactly. Reducing their impact requires both a traceable computational workflow and independent checks that its implementation and scientific assumptions are appropriate.
Terminology should be stated because usage varies. Here, computational reproducibility means obtaining consistent results from the same data and recorded computational procedure; scientific replication means testing the finding with newly collected data or an independently conducted study. model verification asks whether the code implements the intended equations and rules. model validation asks whether those equations and rules are adequate for the scientific use. A result can pass the first two computational checks and still fail biological validation. Conversely, floating-point differences between platforms need not invalidate a conclusion if they lie within a justified numerical or statistical tolerance.
A software bug can enter at every stage. A sample-order mismatch may associate measurements with the wrong labels while retaining plausible matrix dimensions. An indexing or sign error can reverse a treatment contrast. Unit conversion errors can alter a kinetic rate by orders of magnitude. A mistaken stopping rule can report an optimizer's initial value as its solution. A preprocessing script can leak test-set information into training and exaggerate predictive accuracy. These failures are dangerous because an apparently reasonable plot and a successful program exit are weak tests of correctness.
Data handling provides a documented biological example. Spreadsheet defaults can coerce identifiers into dates or numbers: historical gene symbols such as SEPT2 and MARCH1 are examples of this problem. A survey found such conversions in published supplementary gene lists, demonstrating that silent changes can propagate beyond the originating analysis.[ref]
Gene name errors are widespread in the scientific literature by Mark Ziemann and others. Abstract; examples of gene-symbol conversion and manual verification of affected supplementary files
Automatic conversion may be intended spreadsheet behavior, but it becomes a defect in an analysis that requires exact textual identifiers. Specifying identifier columns as strings, preserving stable accessions, checking uniqueness and matching against an expected identifier set directly address that failure mode.A second example distinguishes coding errors from modeling errors. The AFNI 3dClustSim program had a defect in variance rescaling after smoothing, associated with boundary effects. Its maintainers documented the repair and assessed its consequences. Their analysis also showed why the broader problem of inflated cluster-inference false positives could not be attributed to that defect alone: assumptions about spatial autocorrelation and the inferential procedure mattered as well.[ref]
FMRI Clustering in AFNI: False-Positive Rates Redux by Robert W. Cox and others. Results, “The past, I: 3dClustSim, and ‘The Bug’”; Discussion, effect of the correction
Thus correcting an implementation is necessary but does not automatically validate the statistical method it implements.The first defense is a clear, inspectable specification. Write down units, array meanings, intended inputs and limiting behavior; divide a pipeline into small components with explicit interfaces. A unit test for a decay solver can compare against . An integration test should check the connection between data import, model fitting and plotting, including sample identifiers and units. regression tests preserve justified behavior after a fix, but a saved output from an unexamined older version is not an independent truth standard. Test quality matters more than the percentage of lines executed.
Exact reference answers are often unavailable, so property-based testing and metamorphic testing are valuable. A closed chemical conversion should conserve total material; a probability vector should be nonnegative and sum to one; consistently permuting independent sample rows should leave a deterministic fitted model unchanged. A valid metamorphic relation must match the problem: permuting a time series would generally destroy temporal information rather than preserve the expected result. Testing extreme inputs, empty groups and zero-rate cases often exposes bugs that ordinary examples miss. Code review and an independently written implementation can find shared blind spots that additional tests derived from the same erroneous formula cannot.[ref]
Best Practices for Scientific Computing by Greg Wilson and others. sections on modular development, testing, assertions and collaborative code review
Numerical algorithms introduce another layer. For explicit Euler integration of , the update is . The exact solution is positive and decays, but makes the discretization unstable, and even produces alternating negative values. This can be an unsuitable algorithm or step size rather than a typographical defect. Comparing step sizes, solver tolerances and alternative algorithms separates discretization error from a biological prediction. In a sensitive dynamical system, convergence of statistical summaries may be more meaningful than exact agreement of a long trajectory.
Traceability makes failures discoverable and corrections usable. version control identifies the code revision that generated a result; data provenance records raw inputs, reference annotations, filtering decisions and transformations. An executable workflow regenerates outputs from those inputs without undocumented manual edits. Dependency versions and a recorded environment, possibly a software container, reduce accidental differences between installations. A random seed should be recorded together with the generator and, where relevant, the allocation of streams to parallel workers. One seed is not an uncertainty estimate, and parallel scheduling or library changes can defeat bitwise repetition even when it is reused.
continuous integration can run focused tests whenever the analysis changes, while release notes and archived versions let readers distinguish corrected results from earlier ones. The environment archive preserves the ability to reproduce an old computation; the maintained version should also contain the scientific correction. Publishing only a container that reliably reproduces a wrong result is inadequate. When a defect is found, quantify which conclusions change, rerun affected analyses, retain a minimal reproducing example and communicate the correction through the appropriate scientific record. Reliable computational science combines reproducible execution, verified implementation, quantified numerical error and validation against evidence; none is a substitute for the others.
Research transparency Created 2026-10-24 Updated 2026-11-05
Making the provenance, design, measurements and analytical choices underlying a research claim sufficiently inspectable for informed evaluation. It can support computational reproducibility and scientific replication, but does not by itself establish that the measured variable represents the intended construct.
Scientific reproducibility Created 2026-10-20 Updated 2026-11-05
Scientific reproducibility concerns whether the basis of a scientific result can be reconstructed and checked. It requires enough information about the measurement, data handling and reasoning to distinguish repeating a procedure from merely obtaining a similar-looking answer.
Terminology varies between fields. The 2019 National Academies report distinguishes computational reproducibility, using the same data and computational procedure, from scientific replication, which collects new data to address the same scientific question. These test different links in the argument. Repeating a calculation can expose a software or transcription error; new observations can test whether the finding persists beyond the original sample.[ref]
Reproducibility and Replicability in Science by National Academies of Sciences, Engineering, and Medicine. Summary, pp. 6–9: Definitions, Reproducibility and the distinction from new-data replication
Consider an original toy measurement of three lengths, , and units. Recomputing their arithmetic mean as checks the recorded calculation. Measuring three new objects tests a different question: whether the estimated mean describes further observations, allowing for measurement uncertainty and sampling variation. Neither exercise alone establishes that the instrument was correctly calibrated.