For the Benjamini-Hochberg procedure, replace a true-null p-value by zero and call the new rejection count . The Benjamini-Hochberg leave-one-out identity identifies rejection with with the event and . If is uniform and independent of all the other p-values, each possible rejection count contributes to its expected false-discovery fraction. Summation gives per true null and the displayed exact false discovery rate. Super-uniform independent nulls yield the corresponding upper bound.
Past exam of the mathematics course of the University of Cambridge 2013 iii Paper 31 4 Solution Created 2026-10-03 Updated 2026-10-07
Let be the indices of the true null hypotheses, let , and let count false rejections. The familywise error rate isThe Bonferroni correction rejects a null exactly when its p-value is at most . The union bound givesThis controls the familywise error rate for every configuration of true and false nulls, with arbitrary dependence between their p-values. Marginal super-uniformity, , would also suffice.
For the Holm step-down procedure, inspect the ordered p-values in ascending order and stop at the first failed comparison. If the first comparison fails, reject none; this is the convention when the defining set is empty. Ties can be ordered by any fixed rule.
If there can be no false rejection. Otherwise let be the rank of the first true null. Since at most false nulls precede it, and thus . If any true null is rejected, the step-down rule must have rejected this first true null, which requiresUsing the union bound on the true-null p-values provesThis first true null argument for Holm control also requires no independence. Holm controls the familywise error rate at level under arbitrary dependence.
Now let count all rejections. The false discovery rate is the expected proportion of rejections that are false, with zero assigned when nothing is rejected:It differs from the familywise error rate: several false rejections can still represent a small proportion of a large collection of discoveries.
The BH procedure is a step-up rule. SetIf , reject nothing; otherwise reject all hypotheses with . Exactly hypotheses are rejected: if more than p-values were below that threshold, the next ordered value would also satisfy its own larger threshold, contradicting maximality. Thus . Unlike the step-down rule, a failed early comparison does not make this procedure stop.
For the proof, assume that each true-null p-value is uniform on and independent of the entire vector of the other p-values. Joint independence of all p-values is a sufficient condition; the false-null marginal distributions can be arbitrary. Mere uniformity of the true-null marginals without a dependence condition is insufficient for this argument or for general unmodified BH control.
Fix a true null . Replace its p-value by zero and let be the number of rejections made by the Benjamini-Hochberg procedure on the modified vector. This variable depends only on the other p-values, and . The Benjamini-Hochberg leave-one-out identity isTo prove it, suppose is rejected with . Decreasing its p-value to zero leaves every ordered value above rank unchanged, since it was already among the first values. Those higher ranks still fail their thresholds, while rank still succeeds; hence . Conversely, suppose and . Restoring still leaves at least values at most , so . Increasing one p-value cannot increase the maximal successful rank, so . This gives equality and rejection of .
Independence and uniformity now giveSumming over the true nulls proves the exact false discovery rate under independent null p-values:If the independent true-null p-values are only super-uniform, the same calculation gives the inequality instead of equality. The distinction between the two types of error control and the dependence conditions is essential: Bonferroni and Holm have the preceding guarantees without independence, while the BH proof here explicitly uses it.
Past exam of the mathematics course of the University of Cambridge 2017 iii Paper 205 3 Solution Created 2026-10-03 Updated 2026-10-05
Let denote the number of rejected null hypotheses and the number of rejected true null hypotheses. The familywise error rate is , whereas the false discovery rate is , with a zero contribution when . A valid true-null p-value satisfies for .
For the closed testing procedure, choose a level- local test of every nonempty intersection . Reject only if every intersection containing it is rejected by its local test. If any true null hypothesis is rejected, the local test of must reject. Since that intersection is true, its rejection probability is at most . If is empty there can be no false rejection. This proves strong closed-testing control of the familywise error rate, under arbitrary dependence, provided each local test is valid under its whole intersection null.
For the Benjamini-Hochberg procedure, order the p-values and setwith maximum zero if the set is empty; reject the first hypotheses. To prove its false discovery rate bound, fix a true-null index . Replace by zero and let be the modified rejection count. This is a function of the other p-values and satisfies .
The Benjamini-Hochberg leave-one-out identity isHere is its deterministic justification. If was rejected, lowering its value cannot decrease , and leaves all ordered values of ranks greater than unchanged. No such rank can newly satisfy its threshold, so . Conversely, if and , at least other values and are at most this threshold. Therefore the original procedure has . Monotonicity under decreasing also gives , so and is rejected. This establishes the identity; harmless deterministic tie-breaking also covers the artificial zero introduced in the modified data.
The assumed independence makes independent of . Conditional on , validity of the p-value impliesSum over to obtainIf true-null p-values are exactly uniform rather than merely valid, the first inequality is equality. No restriction on dependence among the false-null p-values is needed beyond their joint independence from the true-null values.
For the requested Simes inequality, apply this result to the true-null p-values alone, for . Every rejection is now false, so and the false discovery rate is the probability of any rejection. The event of at least one Benjamini-Hochberg procedure rejection is exactlyIts probability is at most . This holds for ; with no true nulls the written minimum and division by are undefined and the error-control statement is instead vacuous.
The final step-up rule is the Hochberg procedure. Use Simes tests as the local tests for a closed testing procedure: reject when . Under any true intersection all its p-values are valid and independent, so the Simes inequality proves level for its local test. The implication supplied in the question says that whenever the Hochberg procedure rejects , every intersection containing passes its local Simes test. Thus every such rejection is also a closed-testing rejection. ConsequentlyThe true-null independence assumption is essential to this particular proof of local-test validity; the arbitrary-dependence guarantee of closed testing does not create valid local tests automatically.