= Computational reproducibility
<Computational reproducibility> means obtaining consistent results from the same input data, computational procedure and stated conditions. A reproducible analysis identifies the actual inputs and their <data provenance>, the software versions, parameters and relevant environment. Exact bitwise agreement is stronger than agreement within a justified numerical tolerance.<Reproducibility and Replicability in Science by National Academies of Sciences, Engineering, and Medicine>[Summary, pp. 7–9: Recommendation 4-1 and Expected Results from Attempts to Reproduce Research]{ref}
For example, changing the order of a <floating point> sum can change its last few digits through <rounding error>. A useful check states the acceptable <absolute error> or <relative error> before comparing results. Successfully rerunning the same code also reruns any undetected mistake in it, so consistency is evidence about the computational process rather than a proof that the physical model is correct.
Back to article page
