= Differential test
A <differential test> runs implementations intended to realize the same contract on matched <inputs> and compares their observable results. Disagreement reveals an inconsistency without requiring a precomputed answer. Hypothesis explicitly supports comparisons using either a trusted reference or several implementations none of which is fully trusted.<Integrations Reference in the Hypothesis documentation>[equivalent(), reference-oracle and differential-testing examples]{ref}
For example, compare a new <sorting algorithm> with a simple reference on lists containing duplicates, negative values and empty input. Compare results under the specified ordering contract; unspecified differences are not failures. Agreement alone cannot exclude a shared <software bug>. <Floating-point arithmetic> may require a justified numerical tolerance, and comparison of failures as well as successful <outputs> requires an explicit error contract.
Back to article page
