Composite Simpson's rule 2026-10-18
Composite Simpson's rule applies Simpson's rule to consecutive pairs of equal subintervals. Choose a positive even integer , put , and take quadrature nodes for . There are subintervals, two-subinterval panels, and quadrature nodes. Then
The second sum is empty when , recovering one panel. The quadrature weights, before multiplication by , are . Every interior even-indexed quadrature node is an endpoint of two panels, so its two endpoint contributions add to two; every odd-indexed quadrature node is a midpoint with weight four. An odd cannot be covered by these pairs. The displayed formula requires equal spacing.[ref]
For the exponential function on the interval , the exact integral is . With , the formula gives , an overestimate of about . With , it gives , an overestimate of about . The figure shows the intermediate refinements and compares every computed absolute error with the error bound for composite Simpson's rule.
Figure 1.
Computed composite Simpson errors for the exponential function on zero to one approach fourth-order convergence; the dashed curve is the rigorous bound e over 180 n to the fourth power
. Details.
For this exponential function, a geometric series also gives a compact independent formula. With and , summing the panel contributions yields
A Taylor series at gives
The Big O notation describes the remaining terms as tends to zero. Consequently tends to sixteen: the computed ratios increase from about to . Fourth-order truncation error is also obtained by Richardson extrapolation of the composite trapezoidal rule.[ref]
Suppose has a fourth derivative that is a continuous function on the interval , and there. With , the signed quadrature error here is . Then, for some point ,
In particular, a strictly positive fourth derivative makes : composite Simpson's rule overestimates the integral.[ref]
The bound's scale follows by adding the panel errors. A panel of width has signed quadrature error and absolute error at most . There are panels, so the triangle inequality gives . The intermediate value theorem for the continuous fourth derivative replaces the average of the panel values by a single value , giving the signed formula.
The underlying single-panel calculation can be seen through the Peano kernel theorem. On the interval , define the raw Peano kernel , where is integral minus Simpson's rule and is the positive part. Then for and for . It is strictly negative inside the interval and has integral . Thus
The uniform norm bounds the fourth derivative. The polynomial attains equality. Scaling to endpoint spacing multiplies the coefficient by , giving the sharp single-panel bound . The normalized Peano kernel is , so the external factorial must be omitted when using that normalization.
For the exponential function on , , so and the bound is . Its leading actual absolute error is , smaller than the bound because a uniform upper bound on the fourth derivative does not use its variation across the interval. The theorem bounds truncation error with exact function evaluations; sample errors and roundoff error are additional contributions, as described by noise sensitivity of a quadrature rule.
For linear interpolation at zero and , an error in the second sampled value, with the first exact, changes the linear interpolant by . On , its uniform norm is exactly , whereas the derivative change is the constant . With independent absolute bounds on both samples, the value change remains at most and the derivative change is at most , attained for opposite endpoint errors.
Thus bounded absolute sample noise can remain bounded under value interpolation while becoming unbounded under numerical differentiation as . Fixed nonzero sample noise also prevents exact convergence of the value approximation; stability against perturbations and convergence to exact data are different assertions. For a second-sample error with fixed , value convergence requires , while derivative convergence requires in little o notation, in addition to vanishing truncation error.
Numerical analysis Created 2026-09-24 Updated 2026-10-14
Numerical analysis designs algorithms that approximate mathematical answers while controlling cost and error. Its central question is how much of a computed answer can be trusted. Finding a zero of a function, evaluating an integral, solving a linear system, and evolving an ordinary differential equation all demand choices about accuracy, work, and the information available. MIT's numerical-methods course explicitly treats performance and accuracy as joint concerns.[ref]
Three questions organize error analysis. Truncation error measures the approximation introduced by an algorithm even in exact arithmetic; roundoff error comes from finite precision; the condition number measures sensitivity inherent in the problem. Numerical stability asks how the chosen algorithm propagates perturbations. These distinctions suggest different remedies: refine an approximation, change precision or arithmetic, or improve the accuracy of the input.
For example, the square root has relative condition number for : a small relative change in produces about half that relative change in . In contrast, has relative condition number . Changing to changes to about : an input change of about produces an output change of about . This sensitivity exists even with exact arithmetic. MIT's conditioning lecture separates the condition number from roundoff error in precisely this way.[ref]
Numerical differentiation 2026-10-31
Numerical differentiation estimates derivatives from finitely many function values, often by differentiating an interpolation polynomial or using a finite difference. Its error combines truncation error from the formula with errors already present in the sampled values and additional roundoff error. Decreasing the sample spacing can decrease truncation error while increasing the effect of uncertain values. This tradeoff is explained by the ill-posedness of numerical differentiation when only an absolute value-error bound is available.
Take the boundary-fitted grid and assume the standard consistency of a numerical method hypothesis with bounded fourth partial derivatives. Taylor expansion along each coordinate, including a bounded fourth-order remainder, gives the five-point Laplacian truncation error
The numerical equation and exact grid values therefore imply , where with zero boundary values. Its discrete sine transform eigenvectors are , , and its eigenvalues are
The one-dimensional sine vectors form an orthogonal basis, hence so do their tensor products. The least eigenvalue occurs at . Since for , , so the Euclidean operator norm of is at most . There are residual components, and , giving
The factor matters: pointwise second-order consistency of a numerical method becomes first order in the unweighted Euclidean grid norm.
The PDF does not explicitly impose regularity on the exact solution. Without such a hypothesis its unrestricted error claim is false. Here is a counterexample, with a classical solution smooth in the open square and continuous with zero boundary values:
The sampled numerical solution separates as . Its error solves , where and . The one-dimensional residual is . At the first node,
The leading coefficient is strictly negative. Moreover on , so the triangular-average formula for the second difference and concavity of give . Thus the positive part of every residual is at most . The matrix has a nonnegative inverse, and its first diagonal inverse entry is at least , as follows from the nonnegative Neumann expansion of its tridiagonal matrix. The discrete comparison principle bounds the inverse image of the residual's positive part by , while the negative first residual gives . Hence for small . Since , the full Euclidean error is bounded below by a positive constant. It cannot be . Thus the boxed estimate holds for the intended smooth solution class; a regularity condition is genuinely necessary.
The Taylor expansions about are
Adding them cancels the odd powers of and gives
Thus the centred finite difference has truncation error , so .
The identity
shows that is a particular solution. Combining it with the homogeneous solution from part (b) gives
The condition gives , and gives . Hence
The exact initial-value solution is . Since ,
Thus the absolute endpoint error is , consistent with the second-order truncation error of the centred scheme.
Write for the supplied approximation. The asymptotic expansion is
where is Big O notation. At the refined parameter ,
Multiply this identity by nine and subtract the first one. The unknown leading truncation error coefficient cancels:
Thus the Richardson extrapolation estimate is
The two supplied values give
The correction to the finer approximation is . Neglecting the higher-order remainder would also give .
The conclusion is asymptotic: the leading error order improves from two to at least three when both parameter values are in the range of the stated asymptotic expansion. With no bound on the remainder constant, the two finite observations alone do not guarantee that is closer to . Indeed, for any proposed value , the polynomial model
satisfies , and the required asymptotic expansion with true limit . This is why finite observations do not certify Richardson extrapolation accuracy: asymptotic cancellation provides an improved-order estimator, while a numerical error certificate needs quantitative remainder information.
A diameter-only bound also needs the normalization of a Laplacian stencil, beyond its coefficient signs. For example, a single interior row with allows and on a domain of fixed diameter. Thus arbitrary rescaling of the printed coefficients rules out a universal diameter-only estimate.
For the intended normalized Laplacian stencil, assume and for , with , as holds for the standard two-dimensional central stencil. Let . Then . Use the truncation error convention , so ; reversing the convention does not change the absolute-value argument.
The two functions satisfy . On the prescribed boundary , so their maximum boundary values are at most . Apply the discrete maximum principle from part (a), with its boundary-connectivity assumption. Since ,
This is the standard quadratic-barrier error estimate for a discrete Laplacian. The calculation identifies exactly the additional normalization needed for the bound intended by the question.
The given stencil has nonnegative neighbor weights, zero row sums and a path from every unknown node to the Dirichlet boundary. Thus the discrete maximum principle applies, including at the reflected Neumann edge. The barrier is positive throughout the square.
At an ordinary interior node, central differences differentiate this quadratic exactly, giving . At , the reflected horizontal part is
and the vertical part is two. Thus there. Part (c) therefore permits
On the Dirichlet boundary, the largest barrier value is . Consequently
The truncation error at the Neumann edge is divided by the barrier response , producing a contribution of order . Thus its lower local order does not reduce this global maximum-norm error bound.
Use for the sampled exact solution, and let . The Dirichlet boundary conditions imply . The definition of the truncation error gives the exact discrete equation
Subtracting it from the numerical equation therefore gives
The sign on the right is positive because was defined as the difference approximation minus the exact second derivative. Multiply by , sum over the interior nodes and apply discrete summation by parts:
Again . By Young's inequality for products, . Hence
No fourth derivative is needed for this step: the assumed twice continuous differentiability suffices to define and obtain the discrete energy estimate.
Round-off error arises from representing numbers and carrying out arithmetic at finite precision: storing or subtracting nearly equal numbers can lose information even when the mathematical algorithm is exact. Truncation error arises from replacing a continuous or infinite mathematical operation by a finite approximation, such as discarding higher Taylor series terms in a finite difference. Refining a grid usually reduces truncation error but may increase accumulated round-off error; the two have different origins.
Choose equation (1), evaluated by refined composite Simpson's rule. It gives directly as a one-dimensional definite integral with fixed endpoints and a smooth integrand. It therefore avoids root finding altogether. Before evaluating, use the trigonometric identity
This equivalent integrand is nonnegative throughout , avoiding cancellation between the original two terms. Equation (2) instead subtracts two potentially large exponential functions, while equation (3) would ordinarily put a numerical integration inside a root finding loop.
For even , let , , and compute
Then double . The integrand has bounded derivatives of every fixed order on the integration interval; in the fourth-order error regime, the estimated remaining truncation error in is . The factor follows by eliminating the leading term between the two mesh widths.[ref]
Stop when this estimate is comparable to and further refinement shows stagnation at the roundoff error scale. Check that the differences previously decreased consistently with fourth-order convergence; differences alone are not a rigorous error certificate if cancellation or roundoff error dominates. Compensated summation helps keep the summation error near the precision limit. Refining indefinitely after this point increases work without reliably adding meaningful digits. This answers the requested choice and termination strategy without requiring an explicit value of .
Write the computed function values as , where . The computed central finite difference satisfies
The two contributions are respectively evaluation absolute error and truncation error. For , step-size choice for a noisy central derivative minimizes their sum: its derivative is , negative before and positive after
If , the bound tends to zero as and has no positive minimizing step.
Richardson extrapolation 2026-10-14
Richardson extrapolation improves an approximation by combining two resolutions so that their leading truncation errors cancel. The useful surprise is that the size of that leading error need not be known: its scaling under refinement is enough. This lets an existing finite difference method or quadrature rule produce a more accurate answer from two runs, without first deriving a new method from scratch.
Suppose approximates a fixed limit as the positive resolution parameter tends to zero, with an asymptotic expansion
Here is known, is independent of , and denotes Big O notation. The same asymptotic expansion, with the same , must describe both and . Nested grids make it natural to reuse samples, although nesting itself does not establish this error model. Define the linear combination
The weights sum to one, preserving , while . Thus Richardson extrapolation gives : the next available term in the asymptotic expansion determines the improved order. Justin Solomon's numerical analysis text develops this cancellation for finite differences and applies it to numerical integration.[ref]
Geometrically, regard the approximation as a function of . Its first two terms are a straight line . The two computed points lie at positive ; extending the line through them to gives . This is why the operation is extrapolation: it estimates a value outside the sampled parameter interval. The remaining curvature corresponds to the uncancelled terms of the asymptotic expansion.
Worked example: a centered derivative of sine. For the smooth function , the exact derivative at is . The centered difference quotient, a central finite difference, is
The Taylor series of sine gives the signed truncation error
Thus , and . Halving divides the leading truncation error by four. Combining the two centered difference quotients with Richardson extrapolation gives
Indeed, the coefficient becomes ; the coefficient vanishes. This is the even-power case of Richardson extrapolation discussed in Persson's authored lecture notes.[ref]
For and , the actual computations are
The absolute error drops from about for to for , an improvement by roughly a factor of 498. Refining again gives , whose absolute error is about . The refined truncation errors decrease by nearly four for and nearly sixteen for , reflecting their second and fourth orders.
Figure 1.
Centered sine derivative errors: second order before Richardson extrapolation and fourth order after cancellation
. Source. Details.
The logarithmic scale on each axis turns power laws into straight lines. The dashed reference curves are and , the magnitudes of the leading truncation errors. The plotted errors are computed with 70-digit decimal arithmetic, rather than clipped to a display floor; this isolates the asymptotic expansion from roundoff error.
Worked example: integrating an exponential. Approximate the definite integral of the exponential function on :
For numerical integration, the composite trapezoidal rule adds the areas of trapezoids between successive samples; this is the quadrature use of the endpoint average that also occurs in the ODE trapezoidal rule. On equal intervals, with , define
Summing the geometric series gives . Expanding the exponential function near zero yields
The leading truncation error again has , . On the uniform nested grids with and intervals, Richardson extrapolation therefore gives
This linear combination is exactly composite Simpson's rule on the finer grid, whose spacing is . On each coarse panel, an old endpoint receives weight and the new midpoint weight ; shared old endpoints receive weight . These are the usual Simpson's rule weights multiplied by . The equivalence is explained directly in LeMesurier's university notes.[ref]
For two coarse panels and four fine panels,
The absolute errors are about , and , respectively. Reusing the fine-grid samples makes Richardson extrapolation especially attractive here: the resulting Simpson's rule needs no samples beyond those already used by .
Figure 2.
Exponential quadrature on nested grids: trapezoidal areas, reused sample nodes, and the fourth-order Simpson combination
. Source. Details.
When the improvement is justified. Richardson extrapolation requires the stated asymptotic expansion and the correct positive . If the actual leading exponent is , its coefficient after combination is , so the leading truncation error generally remains. Merely knowing does not supply a common coefficient or a higher-order remainder. A discontinuity of a function, a singular endpoint, or unrelated changes between grids can invalidate the assumed asymptotic expansion; refinement alone does not guarantee improvement for every numerical problem.
Roundoff error and errors in the sampled values survive the cancellation of truncation error. If the coarse and fine values have absolute perturbations bounded by , the perturbation in is bounded by ; for this is . In floating-point arithmetic, a centered difference quotient can also amplify sample errors through division by , so an indefinitely smaller need not be more accurate. Persson's notes describe this balance between truncation error and roundoff error.[ref]
The same cancellation idea appears in step-doubling error estimation. For an order- one-step method for an ordinary differential equation, two half steps each contribute local truncation error of order , so their accumulated leading local error has ratio to one full step. This explains the denominator there; the appropriate exponent follows from that local error model, rather than automatically from a generic use of Richardson extrapolation.
Round-off error Created 2026-10-14 Updated 2026-10-20
Round-off error arises when numbers or arithmetic results are rounded to a finite set of representable values. Its scale is associated with machine epsilon, and its effects depend on subsequent operations. Even a formula with zero truncation error can lose information. In decimal arithmetic rounded to three significant digits, becomes and becomes : subtracting the stored numbers gives zero although the exact difference is . The subtraction exposes errors already present in its nearly equal operands.
This example also has a large condition number: independently perturbing the two inputs of can produce a relative amplification . High sensitivity belongs to the input-output problem; numerical stability concerns the arithmetic used to solve it. Conversely, on a common binary system with 53 significant bits and rounding to nearest, evaluates to zero while evaluates to one. Exact arithmetic makes both one: changing summation order changes the accumulated roundoff error. MIT's summation lecture explains why numerical stability and the condition number must be assessed separately.[ref]
Series truncation error 2026-10-14
For a convergent series with sum , the error from keeping only the first terms is the remaining sum . This truncation error exists even with exact arithmetic. The zeta series truncation error gives explicit bounds for positive reciprocal-power terms.
Simpson's rule Created 2026-10-14 Updated 2026-10-18
Simpson's rule approximates a definite integral by integrating the quadratic polynomial obtained by polynomial interpolation at two endpoints and their midpoint. For an interval with , write and . The three quadrature nodes are , and the quadrature rule is
Thus its quadrature weights are . Their sum is , as required for exactness on constant functions. This is the three-node Newton-Cotes closed quadrature; its degree of precision of a quadrature rule is three, even though the interpolating polynomial has polynomial degree at most two.[ref]
For an original worked example, take the cubic polynomial on the interval . Here and , so the function evaluations give
Simpson's rule gives
An antiderivative supplies an independent exact definite integral:
The quadratic polynomial through the same quadrature nodes is . It differs from between the quadrature nodes, but its definite integral agrees exactly. The picture shows why: the signed polynomial interpolation error has two lobes with opposite integrals.
Figure 1.
An original cubic and its quadratic interpolant have the same integral because the two interpolation-error lobes have signed integrals plus and minus 81 over 64
. Details.
Three function evaluations therefore capture more information about an integral than about the whole function. When an antiderivative is unavailable, composite Simpson's rule repeats this inexpensive local construction over a partition of an interval. For a sufficiently smooth function, halving the node spacing reduces its truncation error by approximately a factor of sixteen.
Suppose and each sampled value of has absolute error at most . For a central finite difference with , the triangle inequality and the sharp central-secant derivative error give
For , the derivative of this bound is . It changes sign at , proving the unique minimum, of value . This balance explains why reducing the step indefinitely amplifies sample absolute error even as truncation error decreases.