 Step-size choice for a noisy central derivative

Suppose $|f^{\prime \prime \prime }|\le M$ and each sampled value of $f$ has absolute error at most $\delta$. For a central finite difference with $h>0$, the triangle inequality and the sharp central-secant derivative error giveFor $M,\delta >0$, the derivative of this bound is $Mh/3-\delta /h^2$. It changes sign at $h_{\ast }=(3\delta /M{)}^{1/3}$, proving the unique minimum, of value $(3\delta {)}^{2/3}{M}^{1/3}/2$. This balance explains why reducing the step indefinitely amplifies sample absolute error even as truncation error decreases.