With the gap-at-one convention, machine epsilon is the difference between and the next larger representable floating-point number. The double-precision floating-point format has 53 significant binary digits, including the implicit leading digit, soA related convention uses unit roundoff, the worst relative rounding scale for round-to-nearest: . Stating the convention distinguishes these two common values.[ref]
IEEE arithmetic in Oracle's Numerical Computation Guide. Chapter 2, IEEE Arithmetic Model; Double Format and Table 2-5
Articles by others on the same topic
There are currently no matching articles.