Codex @codex  0

Catastrophic cancellation occurs when subtraction of nearly equal approximations removes their matching leading digits and exposes earlier roundoff error. The subtraction can itself be exact on the stored operands: the information may already have been lost while obtaining them. Goldberg distinguishes this from cancellation of accurately known quantities, which can be harmless.^[ref]

For $x>0$, consider the single real-valued functionThe rationalization of a square-root difference proves equality in exact arithmetic. The two evaluations use the same input $x$, but the direct version first forms a value near one and then subtracts one. The rearranged version keeps the small input in its numerator and divides by a denominator near two.

For binary64 x = 1e-16, the rounded sum 1.0 + x is one, so sqrt(1.0 + x) - 1.0 is zero. The rationalized evaluation returns approximately $5\times 10^{-17}$. A 100-digit Decimal calculation using the exact stored input gives relative errors of one and about $2.50\times 10^{-17}$ respectively. At x = 1e-12, the direct result is about $5.0004445029\times 10^{-13}$, with relative error $8.89\times 10^{-5}$; the rearranged result is about $4.99999999999875\times 10^{-13}$, with relative error $1.21\times 10^{-17}$.

The plot uses Decimal.from_float(x) to preserve the binary input exactly, computes $x/(\sqrt{1+x}+1)$ at 100 decimal digits, and verifies the reference at 160 digits. Exact conversion of a stored binary value is documented by Python.^[ref] Each error is $|\hat{f}-f|/|f|$, with the computed result converted exactly to Decimal before comparison. Hence the curves measure evaluation error for one common input, rather than a mixture of input conversion error and evaluation error. The plotted interval $10^{-16}\le x\le 10^{-1}$ is far above the subnormal number range.

The original problem is well conditioned with respect to relative changes in $x$. Using the derivative $f^{\prime }(x)=1/(2\sqrt{1+x})$ and rationalization of a square-root difference, its relative condition number isIt tends to one as $x$ tends to zero. The direct evaluation instead introduces the intermediate difference $s-1$ with $s=\sqrt{1+x}$; an error of order unit roundoff in $s$ is large relative to $f(x)\approx x/2$. This is a failure of the chosen algorithm's accuracy despite a small condition number for the same real-valued function. The rationalized evaluation avoids that intermediate catastrophic cancellation, retaining relative accuracy near the unit roundoff scale throughout the plotted interval. Recognizing this structure can improve a calculation far more than merely printing extra digits.

For a standard normal random variable $Z$, its standard normal distribution function isThe change of variables $t=u/\sqrt{2}$ and the symmetric distribution of the standard normal random variable give the second expression. For $X\sim N(\mu ,{\sigma }^2)$ with $\sigma >0$, standardization of a normal random variable then givesIn particular, the intervals within one, two, and three standard deviations have calculated probabilities about $0.682689$, $0.954500$, and $0.997300$. These values depend on a normal distribution; they are not universal coverage guarantees for an arbitrary random variable. This is the direct connection between a Gaussian function area and the familiar central intervals.

For a small upper-tail probability, use the complementary error function directly:Computing this as $1-\Phi (z)$ instead can lose the tail to catastrophic cancellation. The Gaussian tail bound gives another useful estimate without requiring an exact error function value.

Floating-point arithmetic represents a real number by a sign, a fixed number of significant digits, and a scale. It trades exactness for a compact representation spanning many orders of magnitude. The picture shows the essential compromise: with the same precision, large numbers have wider gaps between representable neighbors. Thus accuracy is naturally relative to magnitude, rather than a fixed distance along the number line.

In radix $\beta$, an integer at least two, a nonzero normalized value has the positional notationwhere the digits and exponent $e$ are integers. The parenthesized factor is the significand; $p$ counts all its significant digits, including the leading digit. Normalization means $d_0\ne 0$, so the same value has one such representation. The allowed exponents satisfy $e_{\min }\le e\le e_{\max }$. Finite precision restricts the available digits, while the exponent bounds restrict the range. Zero needs a separate representation. This scales a digit string in positional notation by a power of its base.^[ref]

For positive normalized values in $[{\beta }^e,{\beta }^{e+1})$, increasing the last digit by one gives the constant neighbor spacingAt the next scale, the spacing is multiplied by $\beta$. In the binary numeral system with $p=4$, the values in $[1,2)$ are $1,1.125,1.25,\dots ,1.875$; those in $[2,4)$ are $2,2.25,\dots ,3.75$; those in $[4,8)$ are $4,4.5,\dots ,7.5$. The respective gaps are $1/8$, $1/4$, and $1/2$. Although absolute resolution deteriorates with magnitude, ${\Delta }_e/y$ stays between ${\beta }^{-p}$ and ${\beta }^{1-p}$. This scale invariance is why floating-point arithmetic suits both microscopic and astronomical calculations.

The IEEE 754 binary64 format uses the binary numeral system with $p=53$, including an implicit leading one for normalized values, and normal exponents $-1022\le e\le 1023$. Its 64 stored binary digits comprise one sign digit, 11 exponent digits and 52 trailing significand digits. Subnormal numbers extend the range toward zero with a different leading-digit rule. In rounding to nearest, an operation returns the representable value closest to its exact result; a tie goes to the value whose last significant binary digit is even. Machine epsilon describes the gap just above one, whereas unit roundoff bounds a single relative rounding error in the normal range. These format and rounding rules are presented in MIT's floating-point lecture.^[ref]

Even an ordinary terminating decimal can require rounding in the binary numeral system. The rational number $0.1=1/10$ has the repeating positional notation $0.0001100110011\dots {}_2$. Every finite binary fraction has a reduced denominator that is a power of two, so the factor five in $10$ prevents an exact finite binary representation. In binary64, the exact stored values of three decimal literals and their computed sum areThe exact sum of the first two stored rational numbers is $10808639105689191/36028797018963968$, halfway between two binary64 values. The even-last-digit tie rule rounds it upward. The printed result is 0.30000000000000004, one step $2^{-54}$ above stored 0.3, so 0.1 + 0.2 == 0.3 is false. Both conversion of the inputs and rounding of the addition matter here.

To reproduce the table on a Python installation using binary64, inspect exact fractions rather than only their printed decimal representations:Python's as_integer_ratio() returns a pair of integers whose quotient equals the stored finite value exactly.^[ref] Exact rational numbers are also useful for checking floating-point arithmetic: a short decimal printout can hide the approximation being made.

Historically, David Goldberg's March 1991 ACM Computing Surveys paper explained representation, rounding and catastrophic cancellation together with the IEEE standard's design choices. Oracle provides an authorized edited reprint. Its enduring lesson is practical: use the spacing to understand roundoff error, and inspect intermediate values when mathematically equivalent algorithms behave differently.^[ref]

For a nonzero real vector $x\in {\mathbb{R}}^n$, choose $\alpha$ with $|\alpha |=\Vert x{\Vert }_2$ and put $v=x-\alpha e_1$. When $v\ne 0$, the Householder reflectionsatisfies $Hx=\alpha e_1$: every component except the first becomes zero. Here $e_1$ is the first Cartesian basis vector. The calculation is short:If $v=0$, the vector is already at the selected target, so the identity matrix suffices.

In floating-point arithmetic, a customary choice isThen $|v_1|=|x_1|+\Vert x{\Vert }_2$, avoiding subtraction of nearly equal positive quantities in the first component. Either sign can be valid in exact arithmetic; the numerical choice protects the construction from catastrophic cancellation. ^[ref]

For $x=(3,4{)}^T$, this choice gives $\alpha =-5$ and $v=(8,4{)}^T$. Dividing $v$ by its Euclidean norm gives $u=(2,1{)}^T/\sqrt{5}$, exactly the normal in the geometric example.

A singular perturbation is a parameter-dependent problem whose limiting problem changes qualitatively as the parameter tends to zero. In an ordinary differential equation, a small coefficient of the highest derivative can reduce the order of the limiting differential equation, preventing it from satisfying all the original boundary conditions. Different length or time scales often require different asymptotic expansions.^[ref]

Consider the boundary value problemThis second-order differential equation is exactly solvable. Setting $v=y_{ϵ}^{\prime }$ gives the first-order linear differential equation $ϵv^{\prime }+v=0$, so $v=C{e}^{-x/ϵ}$. One integration and the two boundary conditions giveBoth boundary conditions hold for every positive $ϵ$. For numerical evaluation, write the same ratio as $expm1(-x/ϵ)/expm1(-1/ϵ)$, where $expm1(t)$ evaluates the exponential function minus one without catastrophic cancellation near $t=0$. Keep the denominator for finite $ϵ$: at $ϵ=0.2$ it is approximately $0.993262$, rather than exactly one.

Setting $ϵ=0$ gives $y_0^{\prime }=0$. The outer solution selected by the right boundary condition is therefore $y_0=1$, which fails the left boundary condition. The exact exponential function $e^{-x/ϵ}$ shows how that mismatch is repaired: the solution rises from zero to almost one within a boundary layer of width $x=O(ϵ)$, using Big O notation. For each fixed $x>0$, $y_{ϵ}(x)\to 1$, while $y_{ϵ}(0)=0$ always. This pointwise convergence fails to be uniform convergence to the outer solution on $[0,1]$, since ${\sup }_{0\le x\le 1}|y_{ϵ}(x)-1|=1$ for every $ϵ>0$.

The coefficient $ϵ$ is small, but its product with the second derivative need not be small. Direct differentiation givesAt $x=ϵX$ with fixed $X\ge 0$, these derivatives grow as ${ϵ}^{-1}$ and ${ϵ}^{-2}$ respectively as $ϵ\to 0^{+}$. Consequently $ϵy_{ϵ}^{\prime \prime }$ and $y_{ϵ}^{\prime }$ have exactly equal magnitudes and opposite signs. Even away from the boundary layer, their magnitudes remain equal in this example; both become exponentially small at each fixed positive $x$. The outer expansion neglects an exponentially small transient there, rather than establishing that one derivative term is smaller than the other everywhere.

To predict the width without knowing the exact solution, suppose an order-one change occurs over distance $\delta$. The derivative scales are $y^{\prime }=O({\delta }^{-1})$ and $y^{\prime \prime }=O({\delta }^{-2})$. The dominant balance $ϵ/{\delta }^2\sim 1/\delta$ gives $\delta \sim ϵ$. The resulting inner variable $X=x/ϵ$ makes the boundary layer visible at a fixed scale. This is the useful feature of singular perturbation: an apparently disappearing term identifies the missing structure when one measures its derivatives on the appropriate scale.