If a quadrature error bound is and the relevant numerical resolution is , equating these scales gives an estimate of the strip count beyond which that truncation guarantee reaches the rounding scale. For the composite Simpson's rule, . This estimate is useful for choosing a sensible order of magnitude, but a worst-case bound need not equal the actual quadrature error.
For a normal floating-point number in base two, the spacing depends on the exponent of the result; half the spacing is a natural final-storage rounding scale. Machine epsilon instead measures the gap above one. Function-evaluation errors, accumulation of summation error, cancellation, and the arithmetic used for the nodes can impose an earlier practical accuracy limit. Thus final storage precision alone cannot give a universal optimal strip count.
Floating-point arithmetic Created 2026-10-14 Updated 2026-10-14
Floating-point arithmetic represents a real number by a sign, a fixed number of significant digits, and a scale. It trades exactness for a compact representation spanning many orders of magnitude. The picture shows the essential compromise: with the same precision, large numbers have wider gaps between representable neighbors. Thus accuracy is naturally relative to magnitude, rather than a fixed distance along the number line.
Eight normalized four-bit binary values in each of three intervals, with gaps doubling from one eighth to one half
. Details. In radix , an integer at least two, a nonzero normalized value has the positional notationwhere the digits and exponent are integers. The parenthesized factor is the significand; counts all its significant digits, including the leading digit. Normalization means , so the same value has one such representation. The allowed exponents satisfy . Finite precision restricts the available digits, while the exponent bounds restrict the range. Zero needs a separate representation. This scales a digit string in positional notation by a power of its base.[ref]
https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html. David Goldberg, What Every Computer Scientist Should Know About Floating-Point Arithmetic, §Floating-point Formats, 1991; Oracle's authorized edited reprint
For positive normalized values in , increasing the last digit by one gives the constant neighbor spacingAt the next scale, the spacing is multiplied by . In the binary numeral system with , the values in are ; those in are ; those in are . The respective gaps are , , and . Although absolute resolution deteriorates with magnitude, stays between and . This scale invariance is why floating-point arithmetic suits both microscopic and astronomical calculations.
The IEEE 754 binary64 format uses the binary numeral system with , including an implicit leading one for normalized values, and normal exponents . Its 64 stored binary digits comprise one sign digit, 11 exponent digits and 52 trailing significand digits. Subnormal numbers extend the range toward zero with a different leading-digit rule. In rounding to nearest, an operation returns the representable value closest to its exact result; a tie goes to the value whose last significant binary digit is even. Machine epsilon describes the gap just above one, whereas unit roundoff bounds a single relative rounding error in the normal range. These format and rounding rules are presented in MIT's floating-point lecture.[ref]
https://ocw.mit.edu/courses/18-335j-introduction-to-numerical-methods-spring-2019/2f313023ae3404bc217a81a31b227170_MIT18_335JS19_lec2.pdf. Per-Olof Persson, MIT 18.335J, Lecture 2: Floating-Point Arithmetic, the IEEE Standard, slides 6–9, 2019
Even an ordinary terminating decimal can require rounding in the binary numeral system. The rational number has the repeating positional notation . Every finite binary fraction has a reduced denominator that is a power of two, so the factor five in prevents an exact finite binary representation. In binary64, the exact stored values of three decimal literals and their computed sum areThe exact sum of the first two stored rational numbers is , halfway between two binary64 values. The even-last-digit tie rule rounds it upward. The printed result is
0.30000000000000004, one step above stored 0.3, so 0.1 + 0.2 == 0.3 is false. Both conversion of the inputs and rounding of the addition matter here.To reproduce the table on a Python installation using binary64, inspect exact fractions rather than only their printed decimal representations:Python's
import sys
assert (sys.float_info.radix, sys.float_info.mant_dig) == (2, 53)
for value in (0.1, 0.2, 0.3, 0.1 + 0.2):
print(repr(value), value.as_integer_ratio())
print(0.1 + 0.2 == 0.3) # Falseas_integer_ratio() returns a pair of integers whose quotient equals the stored finite value exactly.[ref]https://docs.python.org/3.14/library/stdtypes.html#float.as_integer_ratio. Python 3.14 documentation, float.as_integer_ratio
Exact rational numbers are also useful for checking floating-point arithmetic: a short decimal printout can hide the approximation being made.Historically, David Goldberg's March 1991 ACM Computing Surveys paper explained representation, rounding and catastrophic cancellation together with the IEEE standard's design choices. Oracle provides an authorized edited reprint. Its enduring lesson is practical: use the spacing to understand roundoff error, and inspect intermediate values when mathematically equivalent algorithms behave differently.[ref]
https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html. Goldberg, 1991, reprint notice and §§Rounding Error, Cancellation
Near a smooth local minimum with and , Taylor's theorem givesIf evaluated function values have absolute uncertainty of order , differences on that scale cannot reliably identify smaller displacements than order . In ordinary floating-point arithmetic, when and evaluations have relative error of order , this becomesFor unit-scale values and curvature, this is the square root of machine epsilon, rather than machine epsilon itself. It explains why a derivative-free golden-section search can lose reliable comparisons while candidate positions remain representably different.[ref]
One dimensional minimization in the GNU Scientific Library manual. Caveats: loss of the quadratic term when added to a nonzero minimum value
This is a precision limit of the evaluated values, not an invariant property of the minimizer. Accurately subtracting a known constant, obtaining values at higher precision, or locating a zero of an accurately evaluated derivative can change the attainable position accuracy. A zero minimum with relative error controlled on its small values also needs separate analysis.
Machine epsilon Created 2026-10-14 Updated 2026-10-20
Here machine epsilon means the gap from one to the next larger value in normalized floating-point arithmetic:It is a relative resolution scale near one, not the smallest positive representable value and not a fixed absolute error for all magnitudes. In binary64, giveswhere is unit roundoff for rounding to nearest. Some authors instead call their machine epsilon; MIT 18.335J uses this half-gap convention. Always check which definition an error bound uses.[ref]
https://ocw.mit.edu/courses/18-335j-introduction-to-numerical-methods-spring-2019/2f313023ae3404bc217a81a31b227170_MIT18_335JS19_lec2.pdf. Persson, MIT 18.335J, Lecture 2, slides 3, 8–9: the half-gap convention
The gap below one is smaller: the preceding binary64 value is , whereas the next value above one is . Under nearest-even rounding,
1.0 + 2.0**-53 is exactly 1.0: the exact sum is a tie and one has the even final binary digit. By contrast, 1.0 + 2.0**-52 is the next value. Thus searching for the smallest increment that changes one by trial addition also depends on the rounding convention. At the next larger binary64 value is , illustrating why a small absolute increment can disappear at large magnitude.With the gap-at-one convention, machine epsilon is the difference between and the next larger representable floating-point number. The double-precision floating-point format has 53 significant binary digits, including the implicit leading digit, soA related convention uses unit roundoff, the worst relative rounding scale for round-to-nearest: . Stating the convention distinguishes these two common values.[ref]
IEEE arithmetic in Oracle's Numerical Computation Guide. Chapter 2, IEEE Arithmetic Model; Double Format and Table 2-5
Use the standard spacing convention for machine epsilon: the next representable floating-point number above is . In a normalized binary representation this corresponds to significant binary digits, including the leading digit. The spacing throughout is therefore .
With rounding to nearest, the associated unit roundoff is half that spacing, ; for a normal result the relative round-off error is bounded by . Some texts instead call machine epsilon. Here denotes the spacing scale, and the iteration counts below use that convention. It describes representation precision, not a guarantee that an arbitrarily evaluated function has a relative error this small.
Choose (iii), . Here the root has magnitude between and . Machine epsilon measures relative precision near unity; the spacing of representable floating-point numbers near the root is therefore of order . A step tolerance scaled to that magnitude is consistent with the attainable absolute positional accuracy. For IEEE binary64, using , , comparable to the local spacing. A fixed numerical constant cannot promise an exact correctly rounded root, but this is the most sensible of the three proposals.
Condition (ii) asks for a step below , a scale appropriate near one rather than near this root. A nonzero representable change near the root is normally much larger. The test may trigger only after exact stagnation, and rounding-induced oscillation can prevent that; it asks for unnecessarily fine absolute resolution.
Condition (i) measures a residual, not a position error. It changes if the same root problem is multiplied by a constant. Moreover, evaluating can involve cancellation among quantities of order , so an absolute residual of order need not be meaningful or attainable. Here , making residual and position errors comparable in exact arithmetic, but their requested absolute scale is still too small. A practical scale-aware stopping criterion for Newton iteration should also detect stagnation or a cycle rather than assuming exact arithmetic persists indefinitely.
Round-off error Created 2026-10-14 Updated 2026-10-20
Round-off error arises when numbers or arithmetic results are rounded to a finite set of representable values. Its scale is associated with machine epsilon, and its effects depend on subsequent operations. Even a formula with zero truncation error can lose information. In decimal arithmetic rounded to three significant digits, becomes and becomes : subtracting the stored numbers gives zero although the exact difference is . The subtraction exposes errors already present in its nearly equal operands.
This example also has a large condition number: independently perturbing the two inputs of can produce a relative amplification . High sensitivity belongs to the input-output problem; numerical stability concerns the arithmetic used to solve it. Conversely, on a common binary system with 53 significant bits and rounding to nearest, evaluates to zero while evaluates to one. Exact arithmetic makes both one: changing summation order changes the accumulated roundoff error. MIT's summation lecture explains why numerical stability and the condition number must be assessed separately.[ref]
https://ocw.mit.edu/courses/18-335j-introduction-to-numerical-methods-spring-2019/pages/week-2/. Steven G. Johnson, MIT 18.335J, Lecture 3: Floating-Point Summation and Backwards Stability, 2019
Near a well-conditioned simple root, a numerically stable Newton root-finding iteration cannot usually resolve steps far below the local spacing of floating-point numbers. That spacing has relative scale comparable to machine epsilon , motivating the displayed position-based criterion with a modest constant . It combines an absolute scale near zero with a relative scale for large inputs. A residual-only test is changed by rescaling the equation, whereas the root and exact Newton step are unchanged. A tiny step alone is not a universal accuracy certificate: derivative errors, stagnation and cycles require separate consideration.
Subnormal number 2026-10-14
A subnormal number is a nonzero representable value smaller in magnitude than the least positive normalized value . Keep the exponent scale fixed at and allow a zero leading digit: the positive values are for integers . This fills the gap toward zero with constant absolute spacing, at the cost of fewer significant digits. Goldberg describes this gradual approach to zero and why it matters for differences of small numbers.[ref]
https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html. Goldberg, 1991, §Denormalized Numbers
For binary64 the least positive normalized value is , and subnormal numbers are spaced by . With nearest rounding and gradual underflow enabled, an exact result with has absolute rounding error at most . A uniform relative bound by unit roundoff fails here: rounds to zero under the even-last-digit tie rule, giving relative error one. Machine epsilon measures resolution near one; it does not measure the distance from zero to the first positive value.
