 Solution

ID: past-exam-of-the-year-1-of-the-computer-science-course-of-the-university-of-oxford/2019/discrete-mathematics-continuous-mathematics-and-probability-resit/5/b/solution

With the gap-at-one convention, machine epsilon is the difference between

1

and the next larger representable floating-point number. The double-precision floating-point format has 53 significant binary digits, including the implicit leading digit, so

ε_{mach} = 2^{- 52} \approx 2.22 \times 1 0^{- 16} .

A related convention uses unit roundoff, the worst relative rounding scale for round-to-nearest:

u = 2^{- 53} \approx 1.11 \times 1 0^{- 16}

. Stating the convention distinguishes these two common values.^[ref]

 New to topics? Read the docs here!