If any true elementary hypothesis is rejected by a closed testing procedure, then the intersection of all true null hypotheses must also be rejected. Its local test has level , so the familywise error rate is at most under arbitrary dependence.
The closure is the collection of all nonempty intersection hypotheses from the original elementary family, including singletons. It is not a topological closure. The closed testing procedure requires local level- tests for these intersections and rejects an elementary hypothesis only when all intersections containing it reject.
Hochberg procedure Created 2026-10-05 Updated 2026-10-06
Order p-values and reject the first hypotheses, where , with for an empty set. Under independent valid true-null p-values, this step-up procedure controls the familywise error rate. One proof embeds its rejections in the closed testing procedure with Simes tests: every intersection containing a rejected hypothesis passes its local Simes test, so closed-testing control of the familywise error rate applies.
The familywise error rate is the probability of rejecting at least one true null:
A valid p-value is super-uniform under its null: for . The Bonferroni correction rejects if . The union bound gives
This controls the familywise error rate for every configuration of true and false nulls, without any independence assumption on the p-values.
For a nonempty index set , the intersection hypothesis asserts that every null indexed by holds. The closure of a family of statistical hypotheses consists of all these nonempty intersections, including the original hypotheses. For each intersection choose a local test of size at most under . Such tests always exist: for example, within use the Bonferroni correction, rejecting if . More powerful local intersection tests can be substituted whenever valid.
The closed testing procedure rejects an elementary hypothesis only when every local test with rejects its intersection hypothesis. If , the intersection is true. Rejecting any true requires rejection by its local test , so
If , there is no possible false rejection. This proves closed-testing control of the familywise error rate. The same argument applies if intersection hypotheses themselves are reported as rejected: requiring all their supersets to reject includes whenever the reported intersection is true.
Finally, consider the given hierarchical family as a laminar family of sets. Call a set true when , so its entire intersection hypothesis is true. Let be the inclusion-maximal true sets in . Distinct members of are disjoint: if they met, laminarity would make one contain the other, contradicting maximality. Consequently,
Every true set lies in a maximal true . If is rejected by the prescribed adjusted p-value rule, the maximum over its ancestors includes , hence
Thus the event of any false rejection is contained in the union of these events over the disjoint maximal true sets. Using validity of each under its true intersection and the union bound,
Equivalently, with probability at least every rejection is correct. This proves familywise error control for a laminar hypothesis family. It requires neither independence nor the presence of all possible intersections or singletons in . Empty index sets are excluded, as required by the denominator ; if there are no true sets the result is immediate. Adjusted values exceeding one can be truncated at one without changing decisions for .
Let denote the number of rejected null hypotheses and the number of rejected true null hypotheses. The familywise error rate is , whereas the false discovery rate is , with a zero contribution when . A valid true-null p-value satisfies for .
For the closed testing procedure, choose a level- local test of every nonempty intersection . Reject only if every intersection containing it is rejected by its local test. If any true null hypothesis is rejected, the local test of must reject. Since that intersection is true, its rejection probability is at most . If is empty there can be no false rejection. This proves strong closed-testing control of the familywise error rate, under arbitrary dependence, provided each local test is valid under its whole intersection null.
For the Benjamini-Hochberg procedure, order the p-values and set
with maximum zero if the set is empty; reject the first hypotheses. To prove its false discovery rate bound, fix a true-null index . Replace by zero and let be the modified rejection count. This is a function of the other p-values and satisfies .
The Benjamini-Hochberg leave-one-out identity is
Here is its deterministic justification. If was rejected, lowering its value cannot decrease , and leaves all ordered values of ranks greater than unchanged. No such rank can newly satisfy its threshold, so . Conversely, if and , at least other values and are at most this threshold. Therefore the original procedure has . Monotonicity under decreasing also gives , so and is rejected. This establishes the identity; harmless deterministic tie-breaking also covers the artificial zero introduced in the modified data.
The assumed independence makes independent of . Conditional on , validity of the p-value implies
Sum over to obtain
If true-null p-values are exactly uniform rather than merely valid, the first inequality is equality. No restriction on dependence among the false-null p-values is needed beyond their joint independence from the true-null values.
For the requested Simes inequality, apply this result to the true-null p-values alone, for . Every rejection is now false, so and the false discovery rate is the probability of any rejection. The event of at least one Benjamini-Hochberg procedure rejection is exactly
Its probability is at most . This holds for ; with no true nulls the written minimum and division by are undefined and the error-control statement is instead vacuous.
The final step-up rule is the Hochberg procedure. Use Simes tests as the local tests for a closed testing procedure: reject when . Under any true intersection all its p-values are valid and independent, so the Simes inequality proves level for its local test. The implication supplied in the question says that whenever the Hochberg procedure rejects , every intersection containing passes its local Simes test. Thus every such rejection is also a closed-testing rejection. Consequently
The true-null independence assumption is essential to this particular proof of local-test validity; the arbitrary-dependence guarantee of closed testing does not create valid local tests automatically.
Estimate the signal by one-dimensional total variation denoising, which penalizes adjacent differences rather than the constant level:
This is a fused Lasso objective. Its L1 norm penalty makes many adjacent differences exactly zero, producing a piecewise-constant fit. For change-point detection, estimate the locations by . The tuning parameter controls the strength of the penalty; it can be chosen, for example, using a validation criterion appropriate to the noise model.
For a general family of elementary nulls , the closed testing procedure forms every nonempty intersection and gives it a local test of level . Reject precisely when all intersections with are locally rejected. This gives strong control of the familywise error rate: let be the indices of the true elementary nulls. If is empty there is no false rejection. Otherwise, rejecting any with requires rejection of the true intersection . Its local test has rejection probability at most , hence
This proof of closed-testing control of the familywise error rate needs no independence of the tests.
For the interval procedure, a true tested interval is contained in one maximal constant block . If , the intervals over which takes its maximum are a subset of those used for , so . Therefore rejection of a true forces rejection of its containing block. Conversely, a rejected block is itself a rejected true null.
There is a small indexing issue in the PDF: its testing family excludes singleton intervals, while may contain singleton blocks. Put . Every true tested interval lies in one of these blocks, so the exact statement, using only hypotheses defined in the paper, is
Singleton blocks contain no eligible tested interval and can be ignored. Equivalently, one may define their nulls as never rejected.
For each , the term occurs in the maximum defining , giving . A valid p-value is a super-uniform random variable under its null, so for ,
The union bound and disjointness of the constant blocks now give
This establishes weighted interval testing for a piecewise-constant mean even when the interval p-values are dependent. Values do not affect rejection at levels at most one.
The familywise error rate is
The Bonferroni correction rejects when . Since a valid p-value is super-uniform under its null, the union bound gives
No independence assumption is needed.
For the closed testing procedure, form the intersection hypothesis for every nonempty and choose a level- local test for each . Reject exactly when every with is rejected by its local test. If any true is rejected, then the intersection of all true nulls is rejected. Since its local test has significance level ,
This is the closed-testing control of the familywise error rate.
Write . Procedure (A) is the Weighted Bonferroni correction: it rejects when . Therefore
Procedure (B) is the Weighted Holm step-down procedure. Let be the first rank in the ordering whose hypothesis is a true null, and put . If any true null is rejected, the procedure reaches step and
because every true null remains among ranks . Hence
and another union bound gives FWER at most .
At step , procedure (B) divides by the total weight still under consideration, which is no larger than . Its critical values therefore increase as hypotheses are rejected. Moreover, if procedure (A) would reject a hypothesis, every earlier ordered also passes the initial threshold, so procedure (B) reaches and rejects it. Thus (B) contains every rejection of (A) and can make strictly more rejections while retaining the same strong FWER control.