Machine epsilon (${ϵ}_{\mathrm{mach}}={\beta }^{1-p}$)

Here machine epsilon means the gap from one to the next larger value in normalized floating-point arithmetic:It is a relative resolution scale near one, not the smallest positive representable value and not a fixed absolute error for all magnitudes. In binary64, $p=53$ giveswhere $u$ is unit roundoff for rounding to nearest. Some authors instead call $u$ their machine epsilon; MIT 18.335J uses this half-gap convention. Always check which definition an error bound uses.^[ref]

The gap below one is smaller: the preceding binary64 value is $1-2^{-53}$, whereas the next value above one is $1+2^{-52}$. Under nearest-even rounding, 1.0 + 2.0**-53 is exactly 1.0: the exact sum is a tie and one has the even final binary digit. By contrast, 1.0 + 2.0**-52 is the next value. Thus searching for the smallest increment that changes one by trial addition also depends on the rounding convention. At $2^{53}$ the next larger binary64 value is $2^{53}+2$, illustrating why a small absolute increment can disappear at large magnitude.

Near a well-conditioned simple root, a numerically stable Newton root-finding iteration cannot usually resolve steps far below the local spacing of floating-point numbers. That spacing has relative scale comparable to machine epsilon $ϵ$, motivating the displayed position-based criterion with a modest constant $C$. It combines an absolute scale near zero with a relative scale for large inputs. A residual-only test is changed by rescaling the equation, whereas the root and exact Newton step are unchanged. A tiny step alone is not a universal accuracy certificate: derivative errors, stagnation and cycles require separate consideration.