Numerical differentiation ($f^{\prime }(x)\approx \frac{f(x+h)-f(x)}{h}$)

Numerical differentiation estimates derivatives from finitely many function values, often by differentiating an interpolation polynomial or using a finite difference. Its error combines truncation error from the formula with errors already present in the sampled values and additional roundoff error. Decreasing the sample spacing can decrease truncation error while increasing the effect of uncertain values. This tradeoff is explained by the ill-posedness of numerical differentiation when only an absolute value-error bound is available.

Recovering a derivative from values controlled only in the uniform norm is an ill-posed problem: the differentiation map is not continuous in those norms. On $[0,1]$, take $f_k(x)=\sin (kx)/k$. Then $\Vert f_k{\Vert }_{\infty }\le 1/k\to 0$, while $f_k^{\prime }(x)=\cos (kx)$ has uniform norm one for every $k$. Arbitrarily small value perturbations can therefore conceal nonvanishing derivative changes.

Numerical differentiation can still be useful when additional smoothness bounds, a suitable sampling scale, or a regularization method control the admissible changes. The basic noise amplification by differentiation of a linear interpolant gives a finite-data version: endpoint uncertainty of size $\delta$ produces derivative uncertainty of order $\delta /h$ at spacing $h$.

For linear interpolation at zero and $h>0$, an error $\delta$ in the second sampled value, with the first exact, changes the linear interpolant by $\Delta p(x)=\delta x/h$. On $[0,h]$, its uniform norm is exactly $|\delta |$, whereas the derivative change is the constant $\delta /h$. With independent absolute bounds $|{\delta }_0|,|{\delta }_1|\le \delta$ on both samples, the value change remains at most $\delta$ and the derivative change is at most $2\delta /h$, attained for opposite endpoint errors.

Thus bounded absolute sample noise can remain bounded under value interpolation while becoming unbounded under numerical differentiation as $h\to 0$. Fixed nonzero sample noise also prevents exact convergence of the value approximation; stability against perturbations and convergence to exact data are different assertions. For a second-sample error $aϵ(h)$ with fixed $a>0$, value convergence requires $ϵ(h)\to 0$, while derivative convergence requires $ϵ(h)=o(h)$ in little o notation, in addition to vanishing truncation error.