= Extremal calibration for nested confidence families
\b[Extremal calibration for nested confidence families] obtains simultaneous protection for many candidate bounds by checking their most demanding marginal endpoints. It uses the ordering of a nested family rather than treating every candidate as an unrelated comparison. The simultaneous event then also protects a choice made after observing the data.
Consider the detection model $r=qp$, with upper <confidence bounds> $r_U(\alpha)$ decreasing in their failure budget $\alpha$ and lower bounds $q_L(\beta)$ increasing in $\beta$. For a prescribed set of candidate allocations $D$, let $\alpha_*$ and $\beta_*$ be the largest respective budgets, with valid bounds defined at those endpoints. On
$$
r\le r_U(\alpha_*),\qquad q\ge q_L(\beta_*)>0,
$$
every candidate satisfies
$$
p\le\frac{r_U(\alpha_*)}{q_L(\beta_*)}
\le\frac{r_U(\alpha)}{q_L(\beta)},\qquad (\alpha,\beta)\in D.
$$
The <union bound> gives simultaneous coverage at least $1-\alpha_*-\beta_*$. Neither independence of the estimates nor a finite number of candidates is needed for this common-event proof.
For the line segment $\beta=\gamma_0-\alpha$, $a\le\alpha\le b$, the endpoints are $\alpha_*=b$ and $\beta_*=\gamma_0-a$. The protected failure budget is therefore $\gamma_0+(b-a)$. The width of the candidate interval consumes part of the simultaneous budget; it cannot be ignored simply because every point on the segment has the same marginal budget sum.
An original exact-tail example has zero detections in $n=37$ and successful detection of all $m=20$ <measurement calibration> positives. <Binomial distribution> tail inversion gives $r_U(\alpha)=1-\alpha^{1/37}$ and $q_L(\beta)=\beta^{1/20}$. NIST describes the one-sided tail construction and simultaneous error-budget principle.<NIST and SEMATECH e-Handbook of Statistical Methods>[§7.2.4.1, exact one-sided intervals; §7.4.7.3, Bonferroni's method]{ref} For candidates $(0.01,0.03)$ and $(0.02,0.02)$, the bounds are about 0.13946 and 0.12201. Their extremal endpoint combination $(0.02,0.03)$ gives about 0.11956 and at least 95% coverage. It is a tighter bound with the larger combined budget, and its validity follows directly from the endpoint event.
The construction requires the stated nesting and valid endpoint bounds. A family whose order reverses or whose <measurement calibration> changes across candidates needs a new argument. If the lower endpoint bound for detection is zero, the prevalence upper bound is one, rather than an undefined division.
Back to article page
