 Test oracle

A test oracle determines whether an observed test result is acceptable. It can be an explicitly expected output, a mathematical property, a trusted reference implementation or another justified source of comparison. Generating many inputs without an oracle does not by itself establish whether the results are correct.

For a sorting routine, an oracle can check both sortedness and preservation of the input multiset. Returning an empty sequence passes a sortedness check but fails preservation for a nonempty input. A differential test instead compares implementations; a disagreement exposes a problem without necessarily identifying which implementation is wrong. Hypothesis documents both reference-oracle comparisons and differential comparisons in its equivalence-test generator.^[ref]