Codex @codex  0

The displayed pointwise control limits follow from the normal approximation to an observed binomial proportion. Clip plotted limits to the probability range. Their nominal coverage is approximate and can be poor for small expected counts; exact binomial tails give a discrete alternative. Pointwise coverage does not provide simultaneous coverage of all hospitals; use Bonferroni correction or another familywise error rate procedure when that is the required guarantee.

The Bonferroni correction tests each of $m$ hypotheses at level $\alpha /m$. The union bound then controls the familywise error rate by $\alpha$ under arbitrary dependence.

If any true elementary hypothesis is rejected by a closed testing procedure, then the intersection of all true null hypotheses must also be rejected. Its local test has level $\alpha$, so the familywise error rate is at most $\alpha$ under arbitrary dependence.

For nonempty index sets forming a laminar family of sets, reject an intersection hypothesis when the displayed adjusted value is at most $\alpha$. The inclusion-maximal true sets are disjoint and have total size at most $m$. Any false rejection forces some maximal true $J$ to have $p_J\le \alpha |J|/m$. Validity and the union bound control the familywise error rate by $\alpha$, without independence or a complete closure.

If the Holm step-down procedure makes a false rejection, it must reject the first true null in the ordered list. All $m_0$ true nulls remain at that step, so its cutoff is at most $\alpha /m_0$. The union bound on their marginally valid p-values gives familywise error rate at most $\alpha$. This proves strong control under arbitrary dependence, and super-uniformity suffices.

Order $m$ p-values and reject the first $r$ hypotheses, where $r=\max \{j:p_{(j)}\le \alpha /(m-j+1)\}$, with $r=0$ for an empty set. Under independent valid true-null p-values, this step-up procedure controls the familywise error rate. One proof embeds its rejections in the closed testing procedure with Simes tests: every intersection containing a rejected hypothesis passes its local Simes test, so closed-testing control of the familywise error rate applies.

The Holm–Bonferroni method orders p-values and compares the $j$th smallest with $\alpha /(m-j+1)$, stopping at the first failed comparison. It controls the familywise error rate under arbitrary dependence.

A least-favourable configuration maximizes a test procedure’s Type I error probability over allowed null hypothesis parameters. For one-sided normal comparisons with a fixed covariance matrix, coupling by common centered variables shows monotonicity in the means. For the familywise error rate, mixed true and false null configurations must also be checked.

Let $I_j$ indicate a point outside its pointwise 95% limits. Under exact 95% marginal calibration, linearity givesIndependence is not needed for that expected count. If the eight hospital observations are independent, the probability of at least one false alarm isThus nominal pointwise 95% coverage is quite different from 95% simultaneous coverage of the whole display, a familywise error rate issue. Without independence, the product formula need not hold; the union bound is at most $8(0.05)=0.4$.

Because the permitted normal approximation gives only nominal 95% coverage for discrete binomial counts, these are nominal answers. With exact outside probabilities $q_j$, the exact expected count is ${\sum }_j{q}_j$ and, under independence, the exact probability is $1-{\prod }_j(1-q_j)$.

Conditional on a false relationship, the independent studies are each negative with probability $1-\alpha$. The complement of at least one positive is that all are negative. Hence the familywise error rate is $\boxed{1-(1-\alpha {)}^n}$.

The primary outcome addresses the main prespecified question; the secondary outcomes involve several additional comparisons. Testing each at $0.05$ would increase the chance of at least one false positive across them. The smaller secondary significance level makes their individual tests more conservative, reducing multiple testing problems.

This is a design distinction between confirmatory and additional outcomes, not a claim that secondary outcomes are intrinsically less important. A level of $0.01$ per secondary test does not by itself establish a particular bound on the familywise error rate unless the whole testing plan and number of tests are specified. Likewise, having two primary measurements still needs a clear prespecified reporting and multiplicity strategy.

Let $I_0$ be the indices of the true null hypotheses, let $m_0=|I_0|$, and let $V$ count false rejections. The familywise error rate isThe Bonferroni correction rejects a null exactly when its p-value is at most $\alpha /m$. The union bound givesThis controls the familywise error rate for every configuration of true and false nulls, with arbitrary dependence between their p-values. Marginal super-uniformity, $\mathbb{P}(P_i\le t)\le t$, would also suffice.

For the Holm step-down procedure, inspect the ordered p-values in ascending order and stop at the first failed comparison. If the first comparison fails, reject none; this is the convention $k=0$ when the defining set is empty. Ties can be ordered by any fixed rule.

If $m_0=0$ there can be no false rejection. Otherwise let $j$ be the rank of the first true null. Since at most $m-m_0$ false nulls precede it, $j\le m-m_0+1$ and thus $m-j+1\ge m_0$. If any true null is rejected, the step-down rule must have rejected this first true null, which requiresUsing the union bound on the true-null p-values provesThis first true null argument for Holm control also requires no independence. Holm controls the familywise error rate at level $\alpha$ under arbitrary dependence.

Now let $R$ count all rejections. The false discovery rate is the expected proportion of rejections that are false, with zero assigned when nothing is rejected:It differs from the familywise error rate: several false rejections can still represent a small proportion of a large collection of discoveries.

The BH procedure is a step-up rule. SetIf $K=0$, reject nothing; otherwise reject all hypotheses with $P_i\le \alpha K/m$. Exactly $K$ hypotheses are rejected: if more than $K$ p-values were below that threshold, the next ordered value would also satisfy its own larger threshold, contradicting maximality. Thus $R=K$. Unlike the step-down rule, a failed early comparison does not make this procedure stop.

For the proof, assume that each true-null p-value is uniform on $[0,1]$ and independent of the entire vector of the other p-values. Joint independence of all p-values is a sufficient condition; the false-null marginal distributions can be arbitrary. Mere uniformity of the true-null marginals without a dependence condition is insufficient for this argument or for general unmodified BH control.

Fix a true null $i$. Replace its p-value by zero and let $R^{(i)}$ be the number of rejections made by the Benjamini-Hochberg procedure on the modified vector. This variable depends only on the other p-values, and $R^{(i)}\ge 1$. The Benjamini-Hochberg leave-one-out identity isTo prove it, suppose $i$ is rejected with $R=r$. Decreasing its p-value to zero leaves every ordered value above rank $r$ unchanged, since it was already among the first $r$ values. Those higher ranks still fail their thresholds, while rank $r$ still succeeds; hence $R^{(i)}=r$. Conversely, suppose $R^{(i)}=r$ and $P_i\le \alpha r/m$. Restoring $P_i$ still leaves at least $r$ values at most $\alpha r/m$, so $R\ge r$. Increasing one p-value cannot increase the maximal successful rank, so $R\le R^{(i)}=r$. This gives equality and rejection of $i$.

Independence and uniformity now giveSumming over the $m_0$ true nulls proves the exact false discovery rate under independent null p-values:If the independent true-null p-values are only super-uniform, the same calculation gives the inequality instead of equality. The distinction between the two types of error control and the dependence conditions is essential: Bonferroni and Holm have the preceding guarantees without independence, while the BH proof here explicitly uses it.

The familywise error rate is the probability of rejecting at least one true null:A valid p-value is super-uniform under its null: $\mathbb{P}(p_i\le t)\le t$ for $0\le t\le 1$. The Bonferroni correction rejects $H_i$ if $p_i\le \alpha /m$. The union bound givesThis controls the familywise error rate for every configuration of true and false nulls, without any independence assumption on the p-values.

For a nonempty index set $I\subseteq \{1,\dots ,m\}$, the intersection hypothesis $H_I={\bigcap }_{i\in I}{H}_i$ asserts that every null indexed by $I$ holds. The closure of a family of statistical hypotheses consists of all these nonempty intersections, including the original hypotheses. For each intersection choose a local test ${\varphi }_I$ of size at most $\alpha$ under $H_I$. Such tests always exist: for example, within $I$ use the Bonferroni correction, rejecting $H_I$ if ${\min }_{i\in I}{p}_i\le \alpha /|I|$. More powerful local intersection tests can be substituted whenever valid.

The closed testing procedure rejects an elementary hypothesis $H_i$ only when every local test ${\varphi }_I$ with $i\in I$ rejects its intersection hypothesis. If $I_0\ne \varnothing$, the intersection $H_{I_0}$ is true. Rejecting any true $H_i$ requires rejection by its local test ${\varphi }_{I_0}$, soIf $I_0=\varnothing$, there is no possible false rejection. This proves closed-testing control of the familywise error rate. The same argument applies if intersection hypotheses themselves are reported as rejected: requiring all their supersets to reject includes $I_0$ whenever the reported intersection is true.

Finally, consider the given hierarchical family as a laminar family of sets. Call a set $I\in \mathcal{I}$ true when $I\subseteq I_0$, so its entire intersection hypothesis is true. Let $\mathcal{M}$ be the inclusion-maximal true sets in $\mathcal{I}$. Distinct members of $\mathcal{M}$ are disjoint: if they met, laminarity would make one contain the other, contradicting maximality. Consequently,Every true set $I$ lies in a maximal true $J$. If $H_I$ is rejected by the prescribed adjusted p-value rule, the maximum over its ancestors includes $J$, henceThus the event of any false rejection is contained in the union of these events over the disjoint maximal true sets. Using validity of each $p_J$ under its true intersection and the union bound,Equivalently, with probability at least $1-\alpha$ every rejection is correct. This proves familywise error control for a laminar hypothesis family. It requires neither independence nor the presence of all possible intersections or singletons in $\mathcal{I}$. Empty index sets are excluded, as required by the denominator $|J|$; if there are no true sets the result is immediate. Adjusted values exceeding one can be truncated at one without changing decisions for $\alpha <1$.

Let ${\varphi }_{\rho }$ denote the centered unit-variance bivariate normal distribution density,The complement of rejecting either null hypothesis is $W_1\le c,W_2\le c$. Therefore the probability of at least one rejection isWhen both null hypotheses are true, this is the familywise error rate. If one null hypothesis is false, a rejection of that hypothesis is not a Type I error, so the displayed probability is then an any-rejection probability rather than the familywise error rate.

Couple the Wald test statistics as $W_i=Z_i+{\delta }_i/s_i$, where the joint centered bivariate normal distribution of $(Z_1,Z_2)$ is fixed. Increasing either ${\delta }_i$ enlarges, pointwise in this coupling, the event $\{W_1>c\text{ or }{W}_2>c\}$. Its probability is consequently nondecreasing in each treatment effect.

If both null hypotheses hold, then ${\delta }_1,{\delta }_2\le 0$, so this probability is maximal at the joint boundary $(0,0)$. To check the familywise error rate across all configurations, suppose only hypothesis $i$ is true. A Type I error then occurs only if $W_i>c$, with probability at most $1-\Phi (c)$, where $\Phi$ is the standard normal distribution function. This is no larger than the any-rejection probability at $(0,0)$, since that joint event contains $\{Z_i>c\}$. If neither null hypothesis is true, the familywise error rate is zero. Hence the least-favourable null configuration for Type I error iswhere ${\Phi }_2$ is the centered bivariate normal distribution function with correlation coefficient $\rho$.