If a quadrature error bound is and the relevant numerical resolution is , equating these scales gives an estimate of the strip count beyond which that truncation guarantee reaches the rounding scale. For the composite Simpson's rule, . This estimate is useful for choosing a sensible order of magnitude, but a worst-case bound need not equal the actual quadrature error.
For a normal floating-point number in base two, the spacing depends on the exponent of the result; half the spacing is a natural final-storage rounding scale. Machine epsilon instead measures the gap above one. Function-evaluation errors, accumulation of summation error, cancellation, and the arithmetic used for the nodes can impose an earlier practical accuracy limit. Thus final storage precision alone cannot give a universal optimal strip count.
Fixed-point arithmetic Created 2026-10-20 Updated 2026-11-05
Fixed-point arithmetic represents values as integer multiples of a fixed scale . Integer operations on , followed by the required rescaling and rounding, implement arithmetic on these values. Unlike floating-point arithmetic, the spacing is constant throughout the representable range.
For , the stored integer represents . Multiplying two encoded values and gives , so returning to the original scale requires dividing the integer product by with a stated rounding rule. Fixed spacing makes resolution predictable, but the chosen integer width limits range and can cause integer overflow.
Floating-point number Created 2026-10-20 Updated 2026-11-05
A finite floating-point number is a value representable in a specified floating-point representation. For fixed precision and exponent bounds, only finitely many real values are available. Most real numbers therefore require rounding before storage.
In binary arithmetic, is exact but is not: a finite binary fraction has a reduced denominator that is a power of two. This is a property of the representation, even before any arithmetic operation occurs. The associated IEEE encodings additionally include infinities and NaN; NaN is a special arithmetic datum rather than a real value.[ref]
Gary Sullivan 2026-11-05
An original neutral-gray example isolates code range conversion from memory layout. Take eight-bit computer RGB inputs , with codes from 0 to 255. Since the three coefficients in the guide's luma combination sum to one, the combined gray value is regardless of which of its two coefficient sets is selected. For its eight-bit nominal video coding, use
Thus gray 64 becomes , while gray 192 becomes . The separation of the two stored luma codes is 110 rather than 128. These values follow the stipulated input/output ranges and nearest-integer rounding; copying an RGB component directly into the luma slot would implement a different mapping.[ref]
Changing a pixel format can rearrange those same codes without performing this conversion. Conversely, converting every pixel's three components does not itself require chroma subsampling. Keeping these operations separate is a practical contribution of the jointly authored guide; no individual authorship of its formulas is inferred.
The White stratum retains the overall distrust–lower-efficacy pattern, the Asian stratum shows a stronger but irregular version of it, and the Black stratum has an opposite descriptive ordinal direction. This supports replication in the dominant stratum together with possible specification by ethnicity in the elaboration model. Sparse minority tables and different significance labels do not by themselves establish a population statistical interaction.
Each table fixes ethnicity and compares the efficacy conditional distributions across technology responses. The body percentages use the technology-response row total within that stratum. The same labels, coding direction and efficacy order apply as before. The strata contain 77, 57 and 2332 observations; they sum to 2466 and their cells reproduce the overall table exactly. Consequently the comparison is genuinely a stratification of the supplied overall counts, rather than a comparison of unrelated samples.
For Asian respondents, High efficacy is 78.95% among those who strongly disagree with the distrust statement, falling to 23.81% among those who disagree, 18.75% among neutral respondents and 19.05% among those who agree. The strong-disagreement versus agreement contrast is about 59.90 percentage points, based on 15 of 19 versus 4 of 21 respondents. This is a large descriptive contrast, but the rest of the table is not a simple monotone progression: Low efficacy peaks at 47.62% in Disagree, versus 10.53% in Strongly disagree, 18.75% in Neutral and 23.81% in Agree; Medium efficacy exceeds 57% in the last two rows. The striking first row is part of a more irregular categorical pattern.
The Asian Pearson chi-squared test of independence gives , 6 statistical degrees of freedom, . The likelihood-ratio test gives , ; its displayed 0.000 is a rounding convention. Cramér's V is about 0.430, compared with 0.148 overall. The generalized phi coefficient is 0.608 and the contingency coefficient 0.519. Two of twelve expected frequencies are below five, with minimum 4.156. This is a modest sparse-cell warning rather than a reason to delete the observed cells, but asymptotic calibration should still be qualified and could be checked by an exact or fixed-margin Monte Carlo test.
The Asian Goodman and Kruskal's gamma is about 0.323, from 1025 concordances and 524 discordances, so its displayed-order direction agrees with the overall association. Somers' D for efficacy given technology is about 0.226, Kendall's tau-b 0.241 and Stuart's tau-c 0.253. These stronger ordinal summaries do not imply that every individual row transition is monotone. Goodman and Kruskal's lambda is about 0.362 for predicting efficacy from technology and 0.286 in the reverse direction: the technology row changes the modal efficacy prediction sufficiently to reduce its errors by about 36.2%. This is a reduction in modal-classification errors, not a fraction of efficacy variance explained.
For Black respondents, High efficacy is 0 of 6, or 0%, in Strongly disagree; 8 of 24, or 33.33%, in Disagree; 2 of 15, or 13.33%, in Neutral; and 7 of 12, or 58.33%, in Agree. Low efficacy is 16.67%, 25.00%, 20.00% and 8.33%, respectively. The strongest-disagreement versus agreement High-efficacy contrast is therefore percentage points, opposite to the Asian and White contrasts. The zero observed High count in the first row does not prove zero population probability, especially with only six respondents in that row. These percentages are compatible with a reversed tendency, but considerable sampling uncertainty is unavoidable.
The Black Pearson chi-squared test of independence gives , 6 statistical degrees of freedom and . The likelihood-ratio test gives , , also above 5%. Thus neither nominal test rejects at 5%; this does not establish exact independence. Eight of twelve expected frequencies are below five, with minimum about 1.158, making the usual large-sample calibration particularly questionable. An exact conditional analysis or Monte Carlo test would be more defensible for a calibrated inferential claim; no uncomputed simulated p-value is asserted.[ref]
The Black Cramér's V is about 0.308, the generalized phi coefficient 0.435 and the contingency coefficient 0.399. An unsigned magnitude of this size can coexist with a nonsignificant test in a small sample. There are 254 concordances and 457 discordances, giving Goodman and Kruskal's gamma about , Somers' D about , Kendall's tau-b about and Stuart's tau-c about . Their negative signs agree with the reversed displayed-order direction. Goodman and Kruskal's lambda is about 0.107 for efficacy given technology and zero for technology given efficacy. In the latter direction, Disagree remains a modal technology category within every efficacy column, so modal prediction gains nothing despite the changed conditional percentages.
For White respondents, High efficacy decreases from 48.55% in Strongly disagree through 43.10% and 32.03% to 27.45% in Agree. The extreme-row contrast is about 21.11 percentage points, from 151 of 311 versus 188 of 685. Low efficacy is 21.22%, 17.10%, 21.57% and 35.04%, respectively. This closely resembles the overall distribution, including the fact that the Low category is not monotone at every step. The White Pearson chi-squared test of independence gives , 6 statistical degrees of freedom and ; the likelihood-ratio test gives , . All expected frequencies exceed 74, so sparse cells are not the problem for these nominal tests.
The White Cramér's V is about 0.152, the generalized phi coefficient 0.215 and the contingency coefficient 0.210. Goodman and Kruskal's gamma is about 0.236, Somers' D 0.155, Kendall's tau-b 0.162 and Stuart's tau-c 0.167: close to the overall ordinal summaries and in the same direction. The two values of Goodman and Kruskal's lambda are about 0.061 for efficacy given technology and 0.062 for technology given efficacy. They again indicate only a small improvement in modal prediction. The large sample size helps explain why the White test has a much smaller p-value than the smaller strata, without implying that its association magnitude is the largest.
The original figure below plots selected empirical conditional probabilities from all the cells. It gives the opposite small-stratum pattern and the dominant White pattern a common scale; it is not a reproduction of a source figure, an estimated continuous trend or a display of uncertainty intervals.
Figure 1.
Efficacy distributions across technology-distrust responses: contrasting ethnic strata and the pooled sample. Lines only connect the four ordered categories; group sizes and sparse cells limit inference.
Details.
The elaboration model asks how the original association changes when a third variable is held fixed. If the association disappeared throughout the strata, an explanation by the control variable might be plausible, subject to a credible causal ordering. Here it persists in the White group, which supplies 94.57% of the sample, and is stronger on several descriptive measures in the Asian group. Therefore ethnicity does not simply explain away the relationship. The White result is a replication of the broad pattern; the reversed Black direction and the Asian shape suggest descriptive specification or possible effect modification. The output supplies no temporal or causal design that would distinguish an actual causal mechanism from other confounding, measurement differences or sampling variation. It also does not demonstrate a Simpson's paradox in which the association reverses in every stratum: that is not what the displayed tables show.
The predominance of White respondents explains why the pooled pattern resembles theirs. By the law of total probability, each technology row's efficacy distribution is a mixture over ethnicity, using the ethnic proportions within that technology row. Those weights need not equal the overall ethnic proportions. As an optional descriptive comparison, standardization with common weights using the overall ethnic shares gives High-efficacy proportions of about 48.38% in Strongly disagree and 27.90% in Agree, a 20.48-percentage point contrast, compared with the crude 21.69-point contrast. This explicitly defined adjustment leaves the broad contrast intact; it is not a causal estimate and does not turn the three different stratum patterns into one common effect.
Finally, a significant test in one group and a nonsignificant test in another is not itself a test that the groups' associations differ. A formal statistical interaction requires a directly specified heterogeneity test or model on a chosen effect scale. The omnibus tests also address whole categorical distributions, whereas gamma and the High-efficacy contrast describe particular ordered features. Multiple stratum tests create a multiple testing issue if used to select a discovery. The defensible result is an association that survives stratification in the dominant group, with important descriptive heterogeneity that merits a direct, sparse-cell-aware investigation rather than an assertion that ethnicity causes the difference.
The fitted age slope is very small and negative, about efficacy points per year, with no conventional evidence against a zero linear slope: , . The model explains only about 0.0367% of the observed efficacy variation. These results concern a linear association, not proof of no possible age relationship or a causal effect of ageing.
The descriptive statistics report 2466 observed ages and efficacy scores, with no missing values for either variable. Age ranges from 14 to 97 years, with sample mean 52.689 years and sample standard deviation 19.064 years. Efficacy ranges from 0 to 16, with sample mean 9.348 and sample standard deviation 3.176 scale points. The standard deviations use different units and should not be compared as though their numerical ratio measured relative substantive variability. The efficacy mean retains the conflict with Question 1's group summaries; that conflict is not resolved by this separate regression output.
A simple linear regression models , where is efficacy and is age in years. Ordinary least squares chooses the coefficients to minimize the residual sum of squares. From the printed coefficients the fitted line is
Holding to this model, ten additional years correspond to fitted scale points. For example, the fitted mean is about 9.453 at age 20 and 9.261 at age 80, a difference of about points. These are fitted means, not exact predictions of individual scores. The line does not imply that every person's efficacy falls at that rate, nor that cross-sectional age differences identify within-person changes over time.
The intercept is the predicted mean at age zero, about 9.517. Since zero is outside the observed 14–97 age range, its direct interpretation requires extrapolation. The printed standard error of 0.188043 and yield the extremely small displayed p-value for a zero intercept. That test does not test whether age is a useful predictor. Recentring age at its observed sample mean would put the intercept near , the fitted mean at a substantively relevant reference age. This agrees with the rounded overall efficacy mean.
The age standard error is 0.003356 points per year. The coefficient divided by this standard error gives , agreeing with the printed t value. Estimating an intercept and one regression slope from 2466 observations leaves residual statistical degrees of freedom. Under a correctly specified normal linear model, the Student t-distribution provides the stated two-sided p-value. At 5%, or 1%, it does not reject . The p-value is not a 34.2% probability that age has no effect, and nonrejection does not establish a practically negligible effect size.[ref]
A 95% confidence interval calculated from the printed estimate and standard error is approximately
or about points per decade. It includes zero and gives the scale of slopes compatible with these summaries and assumptions. Assessing a practically negligible effect would require an explicit substantive tolerance and an appropriate equivalence test, not simply the fact that . A nonlinear association could also coexist with a near-zero overall linear slope.
The coefficient of determination is , meaning that the fitted line explains about of the sample's centred efficacy sum of squares compared with an intercept-only fit. This is not 36.7%, a fraction of people whose efficacy is predicted correctly, or the proportion of efficacy caused by age. The adjusted coefficient of determination, approximately , is slightly negative because its penalty for estimating the extra coefficient exceeds the tiny improvement in fit. It need not be constrained to be nonnegative and does not mean a negative amount of explained variation.
The overall on 1 and 2464 statistical degrees of freedom, with , tests the same zero-slope hypothesis as the age t test because there is only one predictor. The simple-regression output identities give and ; the printed numbers agree allowing their displayed precision. The signed Pearson correlation coefficient is consequently about . These are related summaries of the same weak linear association, not independent corroborating tests.
The residual standard error of 3.177 points estimates the typical unexplained error scale under the fitted model. It is close to the efficacy sample standard deviation, as expected when the line explains almost none of the variation. It is not a standard error of the slope or a guarantee that each individual's prediction error lies within 3.177 points. The residual summary spans roughly to , with sample median about 0.573 and central quartiles about and . An ordinary least-squares fit with an intercept has zero mean residual, not necessarily zero median residual. These few quantiles cannot establish normality, constant conditional spread, absence of outliers or linearity. The apparent last-decimal discrepancies among rounded error scales and other summary fields do not force an additional contradiction: their rounding intervals admit compatible regression summaries.
The conventional uncertainty calculations assume independent, mean-zero errors conditional on age, an appropriate linear mean, and homoscedasticity; normal errors give the usual exact finite-sample t and F calibration. Large samples can relax some distributional requirements but do not remove confounding, selection bias or dependence. A residual-versus-fitted plot could reveal curvature or changing spread, and a normal probability plot could assess error shape; neither plot can be reconstructed from the printed residual quantiles. A prediction interval for an individual includes residual variation and is wider than a confidence interval for the mean response. The supplied output supports a very weak, imprecisely signed linear association with little predictive gain from age alone, while leaving nonlinear, causal and design-based questions unresolved.
In the high-resolution quantization approximation, a smooth input distribution varies little across one conversion interval, so rounding to the nearest level gives an approximately uniform quantization error . Therefore
The input must stay within the analogue-to-digital converter's range. A large standard deviation alone does not guarantee uniform quantization error: an input concentrated exactly on widely separated conversion levels has zero error. Truncation without a half-step correction also has a different mean and root mean square.
Use the standard spacing convention for machine epsilon: the next representable floating-point number above is . In a normalized binary representation this corresponds to significant binary digits, including the leading digit. The spacing throughout is therefore .
With rounding to nearest, the associated unit roundoff is half that spacing, ; for a normal result the relative round-off error is bounded by . Some texts instead call machine epsilon. Here denotes the spacing scale, and the iteration counts below use that convention. It describes representation precision, not a guarantee that an arbitrarily evaluated function has a relative error this small.
Write . Its derivative is , so on it decreases to and then increases. Its endpoint and turning-point values are
Thus , and the preceding bound becomes
For the rounding estimate, standard binary64 has spacing above one. Balancing this nominal absolute accuracy with the bound gives
Thus a few thousand even strips, roughly , reach the double-precision scale according to this bound. A scale-aware estimate leads to the same order. Indeed,
lies in magnitude between and , where the binary64 spacing is . Balancing the error bound against half that spacing gives about strips.
These are saturation estimates from balancing quadrature truncation and round-off error, not sharp guarantees of the actual rounding plateau. The error bound can be conservative, and evaluation and summation errors can limit accuracy earlier. Merely storing the final result does not specify every source of rounding error.
Pooled-mean compatibility under rounding Created 2026-10-25 Updated 2026-10-31
For a partition of a set of complete sample observations into groups of known sizes and total , its sample mean is the weighted mean of the group sample means. Indeed, dividing by proves the identity. If the printed group means each have absolute rounding error at most , their compatible pooled mean lies in
This follows because all the weights are nonnegative. Equivalently, the pooled absolute rounding error is at most , by the triangle inequality. A printed overall mean must have a rounding interval intersecting this interval. Disjoint intervals prove incompatibility, rather than merely a difference of displayed decimals. This is a necessary consistency check for means of the same complete sample; it cannot silently equate a full-sample mean with a different complete-case mean or establish a unique repair to inconsistent output.
Rounding Created 2026-10-20 Updated 2026-11-05
Rounding replaces a number by a permitted representable value according to a specified rule. Rounding to nearest minimizes absolute error; directed rounding instead selects the next permitted value toward positive infinity, negative infinity or zero. A tie between equally near values needs an explicit convention.
On the decimal grid with spacing , rounds to nearest as , with error . Rounding to nearest has error at most half the grid spacing. In normal floating-point arithmetic, that spacing depends on magnitude, leading to a relative-error bound described by unit roundoff. The IEEE round-to-nearest-even rule resolves ties by an even final significand digit.[ref]
Sub-step estimation with uniform dither Created 2026-10-20 Updated 2026-11-05
Let a constant input be with , between consecutive quantization levels and . Add independent centered uniform dither before nearest-level rounding, and assume that neither code clips.
The upper code is selected exactly when . This subinterval occupies fraction of the uniform distribution's support, so with a Bernoulli random variable of parameter . Thus and . Averaging independent conversions gives
For example, an input one quarter of a step above the lower code produces the upper code one quarter of the time. Its averaged code can estimate a sub-step voltage, with a finite remaining variance. Centering, independence and absence of clipping are assumptions of this calculation; a shifted dither distribution can bias the result.
rounding an unclipped voltage to equally spaced levels of separation leaves error in . If the fractional position within a conversion interval is uniformly distributed, direct integration gives zero mean and expectation value . This is a statistical model for the error, not a claim that the converter adds independent white noise for every possible input.