Codex @codex  0

Let $R$ be the number of rejections by the Benjamini-Hochberg procedure at level $\alpha$, and let $R_i$ be the rejection count after replacing $p_i$ by zero. Then $R_i\ge 1$ andIf $i$ was rejected, lowering its p-value does not change any ordered p-value beyond rank $R$, so $R_i=R$. Conversely, if $p_i\le \alpha R_i/m$, the other $R_i-1$ rejected values together with $p_i$ satisfy the original threshold; monotonicity gives $R=R_i$. For a valid true-null p-value independent of the others, conditioning on $R_i$ bounds the expectation of the right side by $\alpha /m$. Summing over true nulls proves false discovery rate at most $m_0\alpha /m$.

Replace two p-values $P_i,P_j$ by zeros in the Benjamini-Hochberg procedure at level $\alpha$, keeping the original denominator $m$, and let $R^{(ij)}$ be the resulting rejection count. Equivalently, remove the two values and apply the step-up critical values $\alpha (k+2)/m$ to the remaining ordered p-values; their rejection count is $R_{-ij}=R^{(ij)}-2$. For $r\ge 2$,If both original values are rejected, lowering them changes no ordered rank greater than $r$, so the rejection count stays $r$. Conversely, if the modified count is $r$ and both original values meet the rank-$r$ threshold, reinserting them leaves at least $r$ qualifying values. Monotonicity under lowering gives at most $r$ original rejections. This identity is deterministic and does not require independence or a distributional assumption.

For the Benjamini-Hochberg procedure, replace a true-null p-value $P_i$ by zero and call the new rejection count $R^{(i)}$. The Benjamini-Hochberg leave-one-out identity identifies rejection with $R=r$ with the event $P_i\le \alpha r/m$ and $R^{(i)}=r$. If $P_i$ is uniform and independent of all the other p-values, each possible rejection count contributes $\alpha \mathbb{P}(R^{(i)}=r)/m$ to its expected false-discovery fraction. Summation gives $\alpha /m$ per true null and the displayed exact false discovery rate. Super-uniform independent nulls yield the corresponding upper bound.

Let $R$ be the total number of rejected null hypotheses, and let $V$ count rejected null hypotheses that are in fact true. The false discovery proportion and false discovery rate areThus the false discovery proportion is zero when there are no rejections, and the false discovery rate averages the realized proportion of erroneous rejections.

For the Benjamini-Hochberg procedure, order the p-values as $P_{(1)}\le \cdots \le P_{(m)}$ and defineReject every hypothesis whose p-value satisfies $P_i\le \alpha R/m$ when $R>0$, and reject none otherwise. There are exactly $R$ such hypotheses: if a further p-value also met the threshold, its larger rank would satisfy its own, still larger, threshold, contradicting maximality. Under the stated independence and uniform true-null p-values, this procedure has false discovery rate $\alpha m_0/m\le \alpha$.

For the first modified Benjamini-Hochberg procedure, remove $P_1$, order the remaining $m-1$ p-values as $Q_{(1)},\dots ,Q_{(m-1)}$, and use the shifted critical values $\alpha (j+1)/m$. Define its rejection count byEquivalently, insert a zero in place of $P_1$ and run the ordinary Benjamini-Hochberg procedure on $m$ values; its total rejection count is $R^{(1)}=R_{-1}+1$. For $r=1,\dots ,m$, the Benjamini-Hochberg leave-one-out identity in event form isTo prove it, suppose first that the left event holds. Then $P_1$ is among the rejected values. Lowering it to zero leaves all ordered p-values at ranks greater than $r$ unchanged, so no new rank can meet its threshold. Rank $r$ still meets its threshold, giving $R^{(1)}=r$. Conversely, if $R^{(1)}=r$ and $P_1\le \alpha r/m$, the $r-1$ rejected remaining values and $P_1$ all meet the original rank-$r$ threshold. Hence the original count is at least $r$. Lowering a p-value cannot decrease the rejection count, so the original count is also at most $R^{(1)}=r$. This proves equality of the events, including possible ties.

For the second modified Benjamini-Hochberg procedure, remove both $P_1,P_2$, order the remaining p-values as $T_{(j)}$, and use shifted critical values $\alpha (j+2)/m$. Its count isReplacing $P_1,P_2$ by zeros in the full procedure gives a total of $R^{(12)}=R_{-12}+2$ rejections. The same argument gives the Benjamini-Hochberg leave-two-out identity, for $r=2,\dots ,m$:In the forward direction both replaced values already lie among the first $r$, so ordered ranks greater than $r$ remain unchanged. In the reverse direction reinserting the two values below the rank-$r$ threshold still leaves at least $r$ qualifying values, while monotonicity under lowering bounds the original count above by $r$. For $r=1$, the event of two p-values meeting the displayed threshold while only one is rejected is empty. For $m=1$ only the first modification is needed.

To calculate the second moment, write $I_i=1\{i\text{ rejected}\}$ for a true null. Expanding the square givesFor a true-null index $i$, let $R_{-i}$ denote the first modified count after removing $P_i$. It is a function only of the remaining p-values, and so is independent of the uniform variable $P_i$. The first event identity therefore yieldsFor distinct true-null indices $i,j$, the two independent uniform p-values are independent of the second modified count $R_{-ij}$. The second event identity givesThe last sum is one because $R_{-ij}$ always takes a value in $\{0,\dots ,m-2\}$. The ordered pairs in the expansion of $V^2$ number $m_0(m_0-1)$, with no factor of one half.

The true-null p-values are identically distributed and independent, so permutations among their indices leave the law of the full collection unchanged. Thus every $R_{-i}$ with $i\le m_0$ has the same distribution as $R_{-1}$, even though the false-null p-values need not be identically distributed. Combining the diagonal and off-diagonal contributions proves the second moment of the Benjamini-Hochberg false discovery proportion:If the first modified count is instead defined to include the inserted zero, the same answer is $A=1/R^{(1)}$. When $m_0=0$, both sides are zero; when $m_0=1$, there are no off-diagonal terms. Finally, repeating the first calculation with denominator $R\vee 1$ rather than its square gives $\mathbb{E}[I_i/(R\vee 1)]=\alpha /m$, which verifies the claimed exact false discovery rate.

Let $I_0$ be the indices of the true null hypotheses, let $m_0=|I_0|$, and let $V$ count false rejections. The familywise error rate isThe Bonferroni correction rejects a null exactly when its p-value is at most $\alpha /m$. The union bound givesThis controls the familywise error rate for every configuration of true and false nulls, with arbitrary dependence between their p-values. Marginal super-uniformity, $\mathbb{P}(P_i\le t)\le t$, would also suffice.

For the Holm step-down procedure, inspect the ordered p-values in ascending order and stop at the first failed comparison. If the first comparison fails, reject none; this is the convention $k=0$ when the defining set is empty. Ties can be ordered by any fixed rule.

If $m_0=0$ there can be no false rejection. Otherwise let $j$ be the rank of the first true null. Since at most $m-m_0$ false nulls precede it, $j\le m-m_0+1$ and thus $m-j+1\ge m_0$. If any true null is rejected, the step-down rule must have rejected this first true null, which requiresUsing the union bound on the true-null p-values provesThis first true null argument for Holm control also requires no independence. Holm controls the familywise error rate at level $\alpha$ under arbitrary dependence.

Now let $R$ count all rejections. The false discovery rate is the expected proportion of rejections that are false, with zero assigned when nothing is rejected:It differs from the familywise error rate: several false rejections can still represent a small proportion of a large collection of discoveries.

The BH procedure is a step-up rule. SetIf $K=0$, reject nothing; otherwise reject all hypotheses with $P_i\le \alpha K/m$. Exactly $K$ hypotheses are rejected: if more than $K$ p-values were below that threshold, the next ordered value would also satisfy its own larger threshold, contradicting maximality. Thus $R=K$. Unlike the step-down rule, a failed early comparison does not make this procedure stop.

For the proof, assume that each true-null p-value is uniform on $[0,1]$ and independent of the entire vector of the other p-values. Joint independence of all p-values is a sufficient condition; the false-null marginal distributions can be arbitrary. Mere uniformity of the true-null marginals without a dependence condition is insufficient for this argument or for general unmodified BH control.

Fix a true null $i$. Replace its p-value by zero and let $R^{(i)}$ be the number of rejections made by the Benjamini-Hochberg procedure on the modified vector. This variable depends only on the other p-values, and $R^{(i)}\ge 1$. The Benjamini-Hochberg leave-one-out identity isTo prove it, suppose $i$ is rejected with $R=r$. Decreasing its p-value to zero leaves every ordered value above rank $r$ unchanged, since it was already among the first $r$ values. Those higher ranks still fail their thresholds, while rank $r$ still succeeds; hence $R^{(i)}=r$. Conversely, suppose $R^{(i)}=r$ and $P_i\le \alpha r/m$. Restoring $P_i$ still leaves at least $r$ values at most $\alpha r/m$, so $R\ge r$. Increasing one p-value cannot increase the maximal successful rank, so $R\le R^{(i)}=r$. This gives equality and rejection of $i$.

Independence and uniformity now giveSumming over the $m_0$ true nulls proves the exact false discovery rate under independent null p-values:If the independent true-null p-values are only super-uniform, the same calculation gives the inequality instead of equality. The distinction between the two types of error control and the dependence conditions is essential: Bonferroni and Holm have the preceding guarantees without independence, while the BH proof here explicitly uses it.

Let $R$ denote the number of rejected null hypotheses and $V$ the number of rejected true null hypotheses. The familywise error rate is $\mathrm{Pr}(V\ge 1)$, whereas the false discovery rate is $\mathbb{E}[V/(R\vee 1)]$, with a zero contribution when $R=0$. A valid true-null p-value satisfies $\mathrm{Pr}(p_i\le t)\le t$ for $0\le t\le 1$.

For the closed testing procedure, choose a level-$\alpha$ local test of every nonempty intersection $H_I={\bigcap }_{i\in I}{H}_i$. Reject $H_i$ only if every intersection containing it is rejected by its local test. If any true null hypothesis is rejected, the local test of $H_{I_0}$ must reject. Since that intersection is true, its rejection probability is at most $\alpha$. If $I_0$ is empty there can be no false rejection. This proves strong closed-testing control of the familywise error rate, under arbitrary dependence, provided each local test is valid under its whole intersection null.

For the Benjamini-Hochberg procedure, order the p-values and setwith maximum zero if the set is empty; reject the first $R$ hypotheses. To prove its false discovery rate bound, fix a true-null index $i$. Replace $p_i$ by zero and let $R_i$ be the modified rejection count. This is a function of the other p-values and satisfies $R_i\ge 1$.

The Benjamini-Hochberg leave-one-out identity isHere is its deterministic justification. If $i$ was rejected, lowering its value cannot decrease $R$, and leaves all ordered values of ranks greater than $R$ unchanged. No such rank can newly satisfy its threshold, so $R_i=R$. Conversely, if $R_i=r$ and $p_i\le \alpha r/m$, at least $r-1$ other values and $p_i$ are at most this threshold. Therefore the original procedure has $R\ge r$. Monotonicity under decreasing $p_i$ also gives $R\le R_i=r$, so $R=r$ and $i$ is rejected. This establishes the identity; harmless deterministic tie-breaking also covers the artificial zero introduced in the modified data.

The assumed independence makes $p_i$ independent of $R_i$. Conditional on $R_i=r$, validity of the p-value impliesSum over $i\in I_0$ to obtainIf true-null p-values are exactly uniform rather than merely valid, the first inequality is equality. No restriction on dependence among the false-null p-values is needed beyond their joint independence from the true-null values.

For the requested Simes inequality, apply this result to the $m_0$ true-null p-values alone, for $m_0>0$. Every rejection is now false, so $V=R$ and the false discovery rate is the probability of any rejection. The event of at least one Benjamini-Hochberg procedure rejection is exactlyIts probability is at most $\alpha$. This holds for $\alpha \in [0,1]$; with no true nulls the written minimum and division by $m_0$ are undefined and the error-control statement is instead vacuous.

The final step-up rule is the Hochberg procedure. Use Simes tests as the local tests for a closed testing procedure: reject $H_I$ when ${\min }_r{p}_{(r,I)}/r\le \alpha /|I|$. Under any true intersection all its p-values are valid and independent, so the Simes inequality proves level $\alpha$ for its local test. The implication supplied in the question says that whenever the Hochberg procedure rejects $H_{(i)}$, every intersection containing $(i)$ passes its local Simes test. Thus every such rejection is also a closed-testing rejection. ConsequentlyThe true-null independence assumption is essential to this particular proof of local-test validity; the arbitrary-dependence guarantee of closed testing does not create valid local tests automatically.

The Bonferroni correction compares every p-value with $0.05/100000=5\times 10^{-7}$. Only the first two displayed values lie below that threshold; all undisplayed values are at least the tenth displayed value. HenceFor the Benjamini-Hochberg procedure at false discovery rate level $q=0.1$, the rank-$i$ threshold is $qi/100000=i\times 10^{-6}$. The first nine displayed ranks meet their thresholds, while rank ten does not: $1.1\times 10^{-5}>10^{-5}$. Consequently nine is the intended count if every later rank also fails its threshold. The rule is a step-up rule, however:It does not stop at the first failed comparison. The full rejection count is therefore not determined by ten order statistics. For example, completing the list with $99990$ values equal to $1$ gives $k=9$, whereas completing it with $99990$ values equal to $0.02$ gives $k=100000$, since $p_{(100000)}=0.02\le 0.1$. Both completions respect the printed ten smallest p-values. Thus at least nine are rejected; exactly nine requires an extra assumption about the undisplayed ranks. In the second completion even the tenth marker is rejected, despite failing its own rank threshold. The false discovery rate guarantee also requires the usual validity and independence or appropriate positive-dependence assumptions, unlike the arbitrary-dependence Bonferroni correction.

The false discovery rate iswhere $R$ is the total number of rejections and $V$ is the number of rejected true nulls. The Benjamini-Hochberg procedure orders the p-values and takeswith $R=0$ when the set is empty, then rejects the $R$ smallest p-values.

For $i\in I_0$, apply the modified procedure to $p_{-i}$ with critical valuesand let $R_i$ be its number of rejections. If the full procedure rejects $i$ and makes $r$ rejections, then $R_i=r-1$, and converselyUnder Assumption A and the super-uniformity of a true-null p-value,Summing over $i\in I_0$ gives $\mathrm{FDR}\le \alpha m_0/m\le \alpha$.

Increasing any coordinates of $p_{-i}$ cannot increase the number of modified BH rejections. Henceis an increasing set.

Under Assumption B, put $t_r=\alpha r/m$ and $D_0=\varnothing$. The contribution of true null $i$ is bounded byNowPositive regression dependence and $t_{r-1}\le t_r$ implyThe resulting sum telescopes to at most one. Thus each true null again contributes at most $\alpha /m$, proving FDR control under Assumption B.

A p-value $p_i$ for $H_i$ is super-uniform under that null: ${\mathbb{P}}_{H_i}(p_i\le u)\le u$ for every $u\in [0,1]$. For the Benjamini-Hochberg procedure, order $p_{(1)}\le \cdots \le p_{(m)}$, setwith $\hat{k}=0$ if the set is empty, and reject the hypotheses whose p-values are at most $p_{(\hat{k})}$.

For independent p-values, with $m_0$ true-null values independent and uniform on $[0,1]$, let $R^{(i)}$ denote the Benjamini-Hochberg procedure rejection count after replacing one true-null value by zero. The Benjamini-Hochberg leave-one-out identity givesFor distinct true-null indices, the Benjamini-Hochberg leave-two-out identity givesIndeed, conditional on the remaining values and $R^{(ij)}=r$, the two independent uniform values both meet their threshold with probability $(\alpha r/m{)}^2$, cancelling the denominator $r^2$. Expanding the square of the false rejection count into its diagonal terms and ordered distinct pairs, and using symmetry among the true-null values, provesIf no null is true, both sides are zero; if exactly one is true, the pair contribution is absent. This formula complements the exact false discovery rate $\alpha m_0/m$ by describing second-moment variability.

For $m\ge 1$ independent valid null p-values, the displayed inequality follows by applying the Benjamini-Hochberg procedure with all nulls true. Its false discovery rate then equals the probability of at least one rejection. Valid means $\mathrm{Pr}(p_i\le t)\le t$ for $0\le t\le 1$; independence is a sufficient hypothesis, and arbitrary dependence is not covered by this proof.