Solution (source code)

= Solution

Numerous successful tests can justify a fallible belief in a general <hypothesis>, but do not usually entail its truth. If finitely many tested instances satisfy $F(x)\Rightarrow G(x)$, a further untested instance can still fail. The logical gap is the <problem of induction>, not a practical complaint that the sample was merely too small. Appealing to the past success of <inductive reasoning> to establish its universal reliability uses the disputed form of inference itself.<An Enquiry Concerning Human Understanding by David Hume>[section IV, part II]{ref}

The kind of testing matters as much as the count. Repeating one measurement with the same unnoticed systematic bias can add little. Independent apparatus, novel circumstances and predictions on which competing theories disagree can be much more informative. A test deliberately designed to expose a failure is stronger than a loose resemblance noticed after the outcome. Still, <auxiliary hypotheses> and instrument assumptions must be considered when interpreting either success or failure.

<Bayesian confirmation> expresses the distinction quantitatively. For independent trials with success probability $p$, $n$ successes have <likelihood> $p^n$. In a simple comparison between a certain-success hypothesis and a specified alternative $p=q<1$, the <Bayes factor> is $q^{-n}$. This favours the first hypothesis without proving it. Independence, the alternative and the <prior probabilities> are substantive assumptions; repeated correlated results cannot automatically be multiplied as independent evidence.

There is also a difference between a universal statement and an adequate approximation within a domain. Many successful low-speed tests of <Newtonian mechanics> justify its use there, without establishing its unrestricted truth at relativistic <speeds>. A reasonable conclusion is <fallibilism>: well-designed and varied successful tests can warrant acceptance or high confidence, while the scope and remaining uncertainty should stay explicit. If “justified” means deductively certain, finite testing generally does not suffice; if it means rationally supported, it often can.