 Extremal calibration for nested confidence families

Extremal calibration for nested confidence families obtains simultaneous protection for many candidate bounds by checking their most demanding marginal endpoints. It uses the ordering of a nested family rather than treating every candidate as an unrelated comparison. The simultaneous event then also protects a choice made after observing the data.

Consider the detection model $r=qp$, with upper confidence bounds $r_U(\alpha )$ decreasing in their failure budget $\alpha$ and lower bounds $q_L(\beta )$ increasing in $\beta$. For a prescribed set of candidate allocations $D$, let ${\alpha }_{\ast }$ and ${\beta }_{\ast }$ be the largest respective budgets, with valid bounds defined at those endpoints. Onevery candidate satisfiesThe union bound gives simultaneous coverage at least $1-{\alpha }_{\ast }-{\beta }_{\ast }$. Neither independence of the estimates nor a finite number of candidates is needed for this common-event proof.

For the line segment $\beta ={\gamma }_0-\alpha$, $a\le \alpha \le b$, the endpoints are ${\alpha }_{\ast }=b$ and ${\beta }_{\ast }={\gamma }_0-a$. The protected failure budget is therefore ${\gamma }_0+(b-a)$. The width of the candidate interval consumes part of the simultaneous budget; it cannot be ignored simply because every point on the segment has the same marginal budget sum.

An original exact-tail example has zero detections in $n=37$ and successful detection of all $m=20$ measurement calibration positives. Binomial distribution tail inversion gives $r_U(\alpha )=1-{\alpha }^{1/37}$ and $q_L(\beta )={\beta }^{1/20}$. NIST describes the one-sided tail construction and simultaneous error-budget principle.^[ref] For candidates $(0.01,0.03)$ and $(0.02,0.02)$, the bounds are about 0.13946 and 0.12201. Their extremal endpoint combination $(0.02,0.03)$ gives about 0.11956 and at least 95% coverage. It is a tighter bound with the larger combined budget, and its validity follows directly from the endpoint event.

The construction requires the stated nesting and valid endpoint bounds. A family whose order reverses or whose measurement calibration changes across candidates needs a new argument. If the lower endpoint bound for detection is zero, the prevalence upper bound is one, rather than an undefined division.