Machine epsilon (${ϵ}_{\mathrm{mach}}={\beta }^{1-p}$)

Here machine epsilon means the gap from one to the next larger value in normalized floating-point arithmetic:It is a relative resolution scale near one, not the smallest positive representable value and not a fixed absolute error for all magnitudes. In binary64, $p=53$ giveswhere $u$ is unit roundoff for rounding to nearest. Some authors instead call $u$ their machine epsilon; MIT 18.335J uses this half-gap convention. Always check which definition an error bound uses.^[ref]

The gap below one is smaller: the preceding binary64 value is $1-2^{-53}$, whereas the next value above one is $1+2^{-52}$. Under nearest-even rounding, 1.0 + 2.0**-53 is exactly 1.0: the exact sum is a tie and one has the even final binary digit. By contrast, 1.0 + 2.0**-52 is the next value. Thus searching for the smallest increment that changes one by trial addition also depends on the rounding convention. At $2^{53}$ the next larger binary64 value is $2^{53}+2$, illustrating why a small absolute increment can disappear at large magnitude.