Resolution-limit conditioning in finite-input detector calibration (${\kappa }_q={\sigma }_q^2/({\sigma }_q^2-{\sigma }_p^2)$)

Resolution-limit conditioning in finite-input detector calibration is the sensitivity of a small inferred detector width to uncertainty in two nearly equal measured widths. In finite-input correction in pulse-detector calibration, the nonnegative normalized convolution model givesThe signed relative sensitivities areThey diverge as the measured output width ${\sigma }_q$ approaches the known input width ${\sigma }_p$. Subtracting two well-measured squared widths can therefore be a poorly conditioned inference.

For an original example, ${\sigma }_q=1.00\mathrm{ns}$ and ${\sigma }_p=0.99\mathrm{ns}$ give $D=0.0199\mathrm{n}{\mathrm{s}}^2$ and ${\sigma }_h\approx 0.1411\mathrm{ns}$. The two relative sensitivity coefficients are about $50.25$ and $-49.25$. A small independent fractional width error can produce a much larger fractional response-width error; a common multiplicative timebase error instead partly cancels because the coefficients sum to one. Correlations must therefore be retained.^[ref]

Near the boundary, report uncertainty in the variance difference before transforming it. For a chosen interval $D\in [-0.010,0.050]\mathrm{n}{\mathrm{s}}^2$, imposing the physical constraint $D\ge 0$ gives a detector-width interval $[0,\sqrt{0.050}]\mathrm{ns}$, approximately $[0,0.2236]\mathrm{ns}$. A coverage-preserving detector-width bound from a variance interval uses this monotone square-root transformation on the physical intersection. It need not resemble a symmetric error bar around a point estimate. An interval wholly below zero contains no admissible width under this model; replacing it silently with a zero-width result hides the inconsistency.

NIST characterized its detector/cable/oscilloscope chain with an optical pulse much shorter than the receiver response.^[ref] That design avoids subtracting two nearly equal broad widths. The variance identity still requires nonnegative profiles and finite second moments; arbitrary full widths at half maximum and responses with negative ringing need a different model.

A coverage-preserving detector-width bound from a variance interval transforms uncertainty in a nonnegative response variance without imposing a symmetric width error bar. Let $D={\sigma }_h^2\ge 0$ and let a random interval $I(X)$ have stated frequentist coverage for $D$. Define the width confidence set byFor the true width ${\sigma }_h\ge 0$,The events are identical, so their coverage probabilities are identical. The transformation does not create coverage that the original interval lacked. Resolution-limit conditioning in finite-input detector calibration explains why working with the variance difference first is useful near a barely resolved detector response.

For an original explicit measurement model, suppose $\hat{D}$ has a normal distribution with mean $D$ and known standard deviation $s=0.025\mathrm{n}{\mathrm{s}}^2$. A reading $\hat{D}=0.040\mathrm{n}{\mathrm{s}}^2$ gives an approximately 95 percent interval $\hat{D}\pm 1.96s=[-0.009,0.089]\mathrm{n}{\mathrm{s}}^2$. Intersecting with the physical range and taking square roots gives $J=[0,0.2983]\mathrm{ns}$. This is an assumed normal model for measuring $D$, not a claim that any difference of fitted squared widths is automatically normal.

If the entire variance interval is negative, the result is an empty physical confidence set for a detector variance difference. Do not replace its negative upper endpoint by zero and report an apparently exact zero width: that changes the confidence-set construction. An empty realization signals disagreement between that interval and the physical parameter range; it can occur even in a valid procedure on an occasional sample that misses the true value.

NIST distinguishes a standard uncertainty from an interval with a justified confidence level and explains the distributional assumptions behind a coverage factor near two.^[ref] The normal interval above needs its stated model; the monotone transformation proof itself does not. Application to a detector additionally requires the nonnegative-profile convolution assumptions of finite-input correction in pulse-detector calibration; a correct statistical transformation cannot repair an invalid physical variance identity.

An empty physical confidence set for a detector variance difference occurs when the entire reported interval for the inferred response variance lies below its admissible range. In the nonnegative-profile detector model, $D={\sigma }_q^2-{\sigma }_p^2={\sigma }_h^2\ge 0$. An interval $I=[\ell ,u]$ gives a physical width setIt is empty exactly when $u<0$. The coverage-preserving detector-width bound from a variance interval proves why this intersection and transformation retain the original coverage event.

For an original reading, suppose an explicitly normal measurement model for $D$ gives $\hat{D}=-0.030\mathrm{n}{\mathrm{s}}^2$ with known standard deviation $s=0.010\mathrm{n}{\mathrm{s}}^2$. The approximately 95 percent interval $\hat{D}\pm 1.96s=[-0.0496,-0.0104]\mathrm{n}{\mathrm{s}}^2$ contains no nonnegative variance. This is disagreement between that interval and the physical model, not a negative physical detector width or an exact measurement of zero width.

Empty sets can occur occasionally even when the model is correct. If $\hat{D}\sim N(D,s^2)$ and $I=[\hat{D}-zs,\hat{D}+zs]$, thenwhere $\Phi$ is the standard normal cumulative distribution function. At the physical boundary $D=0$ and $z=1.96$, the probability is about $2.5\%$; at $D=s$ it is about $0.154\%$. The boundary probability of an empty detector-width confidence set explains why one empty realization is not a logical disproof of the detector model.

Replacing an empty set by the point zero changes the procedure. It must not be presented as though the original interval precisely determined that point. NIST likewise distinguishes an uncertainty interval from an automatically justified confidence level, emphasizing the distributional assumptions behind coverage factors.^[ref]

In practice, examine the width estimates, their correlations and the physical convolution assumptions, while retaining the actual negative interval in the measurement record. NIST's short-input receiver-response measurement illustrates how an independently characterized input helps resolve detector broadening.^[ref] Neither clipping the estimate nor declaring the response model false follows solely from the sign of one noisy variance difference.

The boundary probability of an empty detector-width confidence set is the chance that a valid uncertainty procedure yields no admissible nonnegative response width when the true variance difference is at its boundary. Under the explicit model $\hat{D}\sim N(D,s^2)$, $D\ge 0$, take the central interval $I=[\hat{D}-zs,\hat{D}+zs]$. The empty physical confidence set for a detector variance difference occurs when its upper endpoint is negative:This probability decreases with $D$, so the largest value over the physical parameter range occurs at $D=0$. If $z$ is the usual two-sided $1-\alpha$ normal critical value, the boundary probability is $\alpha /2$.

At that boundary the other way to miss zero is a strictly positive lower endpoint. The two tail events are disjoint and each has probability $\alpha /2$; the remaining probability $1-\alpha$ produces a confidence set containing zero. The joint probability of empty and strictly positive detector-width intervals therefore accounts for the full two-sided noncoverage, not just the empty cases.

For an original repeated-measurement calculation, use $z=1.96$ and 400 independent measurements of a stipulated zero response-variance difference. The expected number of empty physical sets is approximately ten, with standard deviation $\sqrt{400p(1-p)}\approx 3.12$, where $p=\Phi (-1.96)\approx 0.025$. The count follows a binomial distribution only under the stated independent, common-probability model. Roughly ten additional intervals are expected to lie wholly above zero. Their existence is compatible with approximately 95 percent coverage at the boundary.

For an interior example $D=s$, the empty-set probability falls to $\Phi (-2.96)\approx 0.00154$. Empty results are thus most common near the resolution boundary, where subtracting similar measured variances is already delicate.

NIST explains that interpreting expanded uncertainty as a confidence interval requires distributional assumptions.^[ref] These probabilities assume an actual normal model with known $s$; they are not inferred merely from an error bar labelled standard uncertainty. Correlation, systematic offsets or a misspecified detector response can change the repeated-measurement behavior. Retaining empty results in the record allows that behavior to be assessed rather than hiding it by replacing them with zero-width estimates.

The joint probability of empty and strictly positive detector-width intervals must distinguish outcomes of one measurement from counts across repeated measurements. For one normal variance-difference estimate $\hat{D}\sim N(D,s^2)$, let $I=[\hat{D}-zs,\hat{D}+zs]$. An empty physical width set requires $\hat{D}+zs<0$; a strictly positive interval requires $\hat{D}-zs>0$. Those events are disjoint for that same interval, so their intersection has probability zero.

Writing $d=D/s$ and $\Phi$ for the standard normal cumulative distribution function, the three category probabilities areThe last category is an interval containing zero. At $D=0$ and a central $1-\alpha$ interval, these become $(\alpha /2,\alpha /2,1-\alpha )$, as in boundary probability of an empty detector-width confidence set. For interior $D>0$, the categories describe the relation to zero; they are not themselves the coverage categories for the true positive value.

For $N$ independent repetitions with the same probabilities, the counts $(N_E,N_P,N_Z)$ have a multinomial distribution. For nonnegative integers $e,p,k$ summing to $N$,Although the measurement repetitions are independent, their category counts are not. An empty and a positive indicator cannot both equal one in the same repetition, givingThis negative covariance of empty and positive detector-interval counts follows from the fixed total count.

For an original example, take 20 independent boundary measurements with exact central 95 percent normal intervals. Then $p_E=p_P=0.025$. Each count has mean $0.5$, and their covariance is $-0.0125$. The probability of exactly one empty and one strictly positive interval is $20\times 19\times 0.025^2\times 0.95^{18}$, about $0.094$. It is not zero, unlike the intersection of those two outcomes for a single interval.

NIST's uncertainty guidance emphasizes the assumptions needed to assign confidence levels.^[ref] The joint-count calculation above additionally assumes independent repetitions with a common normal model; shared calibration errors can invalidate that independence even when each marginal interval is computed in the same way.

The negative covariance of empty and positive detector-interval counts arises because a fixed number of measurement intervals is divided into mutually exclusive categories. Let $E_i,P_i$ indicate that interval $i$ is respectively physically empty or strictly positive. The same interval cannot satisfy both, so $E_i{P}_i=0$ andFor $N$ independent repetitions with common probabilities, the counts $N_E={\sum }_i{E}_i$ and $N_P={\sum }_i{P}_i$ therefore haveThe joint probability of empty and strictly positive detector-width intervals gives the underlying multinomial model. The sign describes category-count dependence, not physical antagonism between two detectors.

For an original boundary example, take $N=100$ and $p_E=p_P=0.025$. Each count has variance $100(0.025)(0.975)=2.4375$, while their covariance is $-0.0625$. The total tail-category count has variance $2.4375+2.4375-0.125=4.75$. Treating the two counts as independent would give $4.875$ instead. The covariance is needed even though the individual measurement repetitions are independent.

Conditioning on the total is important. If the number of repetitions is random and independent of the category assignments, the law of total covariance givesAn independent total with a Poisson distribution has equal mean and variance, making this covariance zero; a more variable total can make it positive. This is a different sampling protocol, not a contradiction of the fixed-total formula.

Shared calibration drift is another change of model. Repetitions may then have dependent category indicators, and cross-repetition covariance terms must be added. Overdispersion of detector-interval counts under shared drift can make a simple multinomial analysis inadequate.

NIST's measurement guidance retains common influences in uncertainty propagation and requires assumptions for confidence-level interpretation.^[ref] State whether the repetition total is fixed, how the categories are defined and which calibration influences are shared before interpreting their empirical count covariance.

Overdispersion of detector-interval counts under shared drift is excess count variability caused by a common experimental condition changing the chance of an interval outcome. Let $M$ count empty physical detector-width intervals among $N$ readings. Conditional on a block's calibration state $H$, suppose the readings are independent with empty probability $p(H)$, so $M\mid H\sim \mathrm{Binomial}(N,p(H))$.

Writing $\overline{p}=\mathbb{E}[p(H)]$, the law of total variance givesThe first term is the binomial variance using the mean probability. The second comes from sharing one fluctuating condition across the block. Individual indicators have covariance $\mathrm{Var}(p(H))$ across different readings, even though they are independent after conditioning on $H$.

For an original mixture example, suppose equally likely block conditions have empty probabilities $0.01$ and $0.09$. Then $\overline{p}=0.05$ and $\mathrm{Var}(p(H))=0.0016$. With $N=100$, the expected count is five, but its variance is $4.75+15.84=20.59$, with standard deviation about $4.54$. An independent binomial model with probability $0.05$ would give standard deviation only about $2.18$.

The effective repetition count for detector intervals with shared drift can consequently be much smaller than the number of stored traces. Repeating measurements within one unchanged calibration block does not supply the same information about drift as acquiring independent blocks. Record acquisition grouping and reference reuse before pooling interval categories.

The negative covariance of empty and positive detector-interval counts applies to a fixed-probability multinomial model. Shared drift adds cross-repetition dependence and can change that count covariance as well as each count's variance. In the example, the two probability states are stipulated; a physical model would relate them to the receiver's drift, variance-difference estimator and uncertainty calibration.

NIST retains shared influences in uncertainty propagation and explains that confidence interpretations depend on the assumed distributions.^[ref] Observed overdispersion can motivate checking the block model, but does not by itself identify which physical influence caused it.

The effective repetition count for detector intervals with shared drift is the number of independent repetitions that would give the same variance for an average indicator as the actual dependent repetitions. In overdispersion of detector-interval counts under shared drift, let $X_i$ indicate an empty interval, with common probability $\overline{p}$ satisfying $0<\overline{p}<1$. If the covariance between distinct indicators is $v=\mathrm{Var}(p(H))$, their correlation coefficient isThe parent's law of total variance calculation therefore givesThis is an observable-specific variance equivalence, not a literal count of independent traces or a guarantee of confidence interval coverage. A different statistic or a different dependence pattern may require a different equivalence.

For an original feasible mixture, take $\overline{p}=0.1$ and $\rho =0.1$. These moments can arise when the block probability is $0$ with probability $9/19$ and $0.19$ with probability $10/19$: its variance is $0.009$. With $N=50$, the variance-inflation multiplier is $5.9$ andDoubling the repetitions to $100$ only raises $N_{\mathrm{eff}}$ to about $9.17$. For a fixed positive $\rho$, its large-$N$ limit is $1/\rho$, here $10$.

The experiment must therefore record which traces share a reference or drift state. Acquiring genuinely independent calibration blocks addresses a different limitation from oversampling one block; effective repetition count across independent calibration blocks depends on that grouping. A cluster bootstrap can preserve the observed within-block dependence when blocks are suitable independent resampling units. NIST's uncertainty law explicitly retains shared-input covariance, supporting the need to model reference reuse rather than discard it.^[ref]

Effective repetition count across independent calibration blocks accounts for dependence confined to each acquisition block rather than shared by every stored trace. Suppose block $j$ contains $n_j$ empty-interval indicator random variables, all with common probability $0<p<1$ and within-block correlation coefficient $\rho \ge 0$. Assume different blocks are independent. With $N={\sum }_j{n}_j$, summing within-block covariances givesMatching this to the variance of an average of independent indicators yieldsFor $K$ equal blocks of size $m$, this reduces to $N_{\mathrm{eff}}=Km/[1+(m-1)\rho ]$. The denominator involves the block size, not the total $Km$; the single-block expression in effective repetition count for detector intervals with shared drift must not be applied as though the same drift state affected every block.

For an original comparison, fix $N=120$ and $\rho =0.05$. One block gives $N_{\mathrm{eff}}=120/6.95\approx 17.27$. Six independent blocks of twenty give $N_{\mathrm{eff}}=120/1.95\approx 61.54$. Six very uneven blocks of sizes $(100,4,4,4,4,4)$ instead giveAll three designs store the same number of traces. The difference comes from how often the same fluctuating condition is reused. For fixed $N$, $K$, and positive $\rho$, minimizing ${\sum }_j{n}_j^2$ favors balanced blocks; this motivates optimal allocation of detector repetitions across calibration blocks when block-preparation costs also matter.

Independence must come from the physical acquisition model. Starting another file does not erase a shared clock, reference calibration, or slowly varying environmental influence. NIST's uncertainty law retains shared-input covariance explicitly.^[ref] If drift persists between nominal blocks, those cross-block terms must be added and the displayed formula changes. A cluster bootstrap likewise requires blocks that are suitable independent resampling units.

Optimal allocation of detector repetitions across calibration blocks chooses block sizes using the dependence and acquisition-cost model, not just the desired number of stored traces. For fixed total $N$, number of independent blocks $K$, common success probability, and positive within-block correlation $\rho$, effective repetition count across independent calibration blocks givesMaximizing it is equivalent to minimizing ${\sum }_j{n}_j^2$. If two integer block sizes differ by at least two, transferring one repetition from the larger to the smaller decreases that sum by $2(n_i-n_j-1)>0$. Therefore the optimal integer sizes differ by at most one.

For an original fixed-count example, $N=23$ and $K=5$ give balanced sizes $(5,5,5,4,4)$. At $\rho =0.1$, their effective count is $529/31.4\approx 16.85$. Sizes $(11,3,3,3,3)$ give only $529/36.4\approx 14.53$ under the same assumptions.

More independent blocks can help, but preparing one may consume time. In a continuous planning model let each block cost setup time $h>0$, each repetition cost $\tau >0$, and each block contain $n$ repetitions. With total time $T$, the effective count becomesFor $0<\rho <1$, differentiation gives the interior choice $n_{\ast }=\sqrt{h(1-\rho )/(\tau \rho )}$. Integer block counts, minimum sizes, and hardware restrictions must subsequently be imposed.

For chosen $h=9\mathrm{s}$, $\tau =1\mathrm{s}$, $\rho =0.1$ and $T=180\mathrm{s}$, the continuous optimum $n_{\ast }=9$ is feasible with ten blocks and gives $N_{\mathrm{eff}}=50$. Single-repetition blocks give 18, while two blocks of 81 repetitions also give 18. These are model-based comparisons, not measured apparatus timings.

Unequal block noise or different drift correlations require unequal-noise allocation across calibration blocks. Independence between prepared blocks must also be justified by the reference and environmental model; NIST explicitly retains shared-input covariance.^[ref]

Unequal-noise allocation across calibration blocks distributes repetitions using block-specific variability while keeping the intended estimator fixed. Suppose independent blocks supply means ${\overline{Y}}_j$ and the target estimate is $\hat{\theta }={\sum }_j{w}_j{\overline{Y}}_j$ with specified weights $w_j$. Let one observation in block $j$ have variance ${\sigma }_j^2$ and equicorrelation $0\le {\rho }_j<1$ within that block. ThenThe first sum is a block-specific floor; only the second decreases with repetition count. Changing the estimator weights merely to improve this calculation could change the estimand, so they remain fixed here.

For continuous positive allocations with positive costs $c_j>0$, budget ${\sum }_j{c}_j{n}_j=C$ and $a_j=w_j^2{\sigma }_j^2(1-{\rho }_j)>0$, a Lagrange-multiplier calculation givesFor equal observation costs, repetitions are proportional to $|w_j|{\sigma }_j\sqrt{1-{\rho }_j}$, not to the variance itself. Minimum counts and integer rounding must subsequently respect the actual acquisition constraints.

For an original two-block example, take $w_1=w_2=1/2$, equal costs, $C=100$, independent readings within each block, and standard deviations $1$ and $3$ in the output unit. The optimal counts are $25$ and $75$, giving variance $0.04$ instead of $0.05$ from fifty each. If both within-block correlations are instead $0.1$, the same proportions give variance $0.286$, while equal counts give $0.295$. The common-error floor $0.25$ now dominates and cannot be eliminated by taking more observations inside those same blocks.

This extends optimal allocation of detector repetitions across calibration blocks, whose balanced solution used homogeneous noise. For binary detector indicators, the per-reading variance is $p_j(1-p_j)$; differing probabilities require a clearly specified weighted target. The planning variances and correlations must come from an appropriate measurement calibration model, not merely from counting files as independent blocks. NIST's uncertainty propagation retains shared-input covariance for exactly this reason.^[ref]