Past exam of the mathematics course of the University of Cambridge / 2017 / iii / Paper 205 / 3 / Solution

Let $R$ denote the number of rejected null hypotheses and $V$ the number of rejected true null hypotheses. The familywise error rate is $\mathrm{Pr}(V\ge 1)$, whereas the false discovery rate is $\mathbb{E}[V/(R\vee 1)]$, with a zero contribution when $R=0$. A valid true-null p-value satisfies $\mathrm{Pr}(p_i\le t)\le t$ for $0\le t\le 1$.

For the closed testing procedure, choose a level-$\alpha$ local test of every nonempty intersection $H_I={\bigcap }_{i\in I}{H}_i$. Reject $H_i$ only if every intersection containing it is rejected by its local test. If any true null hypothesis is rejected, the local test of $H_{I_0}$ must reject. Since that intersection is true, its rejection probability is at most $\alpha$. If $I_0$ is empty there can be no false rejection. This proves strong closed-testing control of the familywise error rate, under arbitrary dependence, provided each local test is valid under its whole intersection null.

For the Benjamini-Hochberg procedure, order the p-values and setwith maximum zero if the set is empty; reject the first $R$ hypotheses. To prove its false discovery rate bound, fix a true-null index $i$. Replace $p_i$ by zero and let $R_i$ be the modified rejection count. This is a function of the other p-values and satisfies $R_i\ge 1$.

The Benjamini-Hochberg leave-one-out identity isHere is its deterministic justification. If $i$ was rejected, lowering its value cannot decrease $R$, and leaves all ordered values of ranks greater than $R$ unchanged. No such rank can newly satisfy its threshold, so $R_i=R$. Conversely, if $R_i=r$ and $p_i\le \alpha r/m$, at least $r-1$ other values and $p_i$ are at most this threshold. Therefore the original procedure has $R\ge r$. Monotonicity under decreasing $p_i$ also gives $R\le R_i=r$, so $R=r$ and $i$ is rejected. This establishes the identity; harmless deterministic tie-breaking also covers the artificial zero introduced in the modified data.

The assumed independence makes $p_i$ independent of $R_i$. Conditional on $R_i=r$, validity of the p-value impliesSum over $i\in I_0$ to obtainIf true-null p-values are exactly uniform rather than merely valid, the first inequality is equality. No restriction on dependence among the false-null p-values is needed beyond their joint independence from the true-null values.

For the requested Simes inequality, apply this result to the $m_0$ true-null p-values alone, for $m_0>0$. Every rejection is now false, so $V=R$ and the false discovery rate is the probability of any rejection. The event of at least one Benjamini-Hochberg procedure rejection is exactlyIts probability is at most $\alpha$. This holds for $\alpha \in [0,1]$; with no true nulls the written minimum and division by $m_0$ are undefined and the error-control statement is instead vacuous.

The final step-up rule is the Hochberg procedure. Use Simes tests as the local tests for a closed testing procedure: reject $H_I$ when ${\min }_r{p}_{(r,I)}/r\le \alpha /|I|$. Under any true intersection all its p-values are valid and independent, so the Simes inequality proves level $\alpha$ for its local test. The implication supplied in the question says that whenever the Hochberg procedure rejects $H_{(i)}$, every intersection containing $(i)$ passes its local Simes test. Thus every such rejection is also a closed-testing rejection. ConsequentlyThe true-null independence assumption is essential to this particular proof of local-test validity; the arbitrary-dependence guarantee of closed testing does not create valid local tests automatically.