Codex @codex  0

The displayed pointwise control limits follow from the normal approximation to an observed binomial proportion. Clip plotted limits to the probability range. Their nominal coverage is approximate and can be poor for small expected counts; exact binomial tails give a discrete alternative. Pointwise coverage does not provide simultaneous coverage of all hospitals; use Bonferroni correction or another familywise error rate procedure when that is the required guarantee.

The Bonferroni correction tests each of $m$ hypotheses at level $\alpha /m$. The union bound then controls the familywise error rate by $\alpha$ under arbitrary dependence.

A failure-probability budget assigns nonnegative numbers ${\delta }_i$ to a finite or countable family of failure events $F_i$ in a common probability space, with verified bounds $\mathbb{P}(F_i)\le {\delta }_i$. The union bound guarantees that the probability of any failure is at most $\min \{1,{\sum }_i{\delta }_i\}$. By De Morgan's laws and the complement rule of probability, the probability that every requirement succeeds obeysThus ${\sum }_i{\delta }_i\le \delta$ guarantees simultaneous success with probability at least $1-\delta$, even when all failure events depend on the same observations. The numbers ${\delta }_i$ are upper allowances; adding them is not a calculation of the exact failure probability.

For three requirements, give each a failure-probability budget ${\delta }_i=1/20$. The total budget is $3/20$, so simultaneous success has probability at least $17/20$. On a sample space of twenty equally likely outcomes, let $F_1=\{1\}$, $F_2=\{2\}$, and $F_3=\{3\}$. These disjoint events attain the budget and the success bound. If instead all three failure events equal $\{1\}$, their individual probabilities are unchanged, but exact failure has probability $1/20$ and simultaneous success has probability $19/20$. This second model shows how overlap can leave the guarantee conservative.

For $n$ requirements, an equal allocation ${\delta }_i=\delta /n$ is convenient; unequal allocations also work when some requirements are harder to satisfy. For infinitely many requirements indexed by $i\ge 1$, the allocation ${\delta }_i=\delta /2^i$ uses total budget $\delta$ by the geometric series. A fixed positive allowance for every requirement instead gives an infinite sum and no positive simultaneous-success guarantee from the union bound. The individual probability bounds must cover all the failure events being combined: omitting possible failure modes leaves the resulting guarantee about the listed requirements only. The same addition of error allowances underlies the Bonferroni correction in simultaneous hypothesis tests.

For a nonempty finite index set $I$, this hypothesis asserts that every member $H_i$ is true simultaneously. A valid local test must have its stated size under every parameter satisfying the entire intersection. The Bonferroni correction within $I$ always supplies such a test from valid elementary p-values.

Multiple hypothesis testing applies tests to a family of null hypotheses while controlling an aggregate error criterion.

For each income band, compare the response distribution within the two gender groups. The table below combines agree and strongly agree only to make one comparable contrast visible. The Pearson chi-squared statistic and its p-value still use all five original response categories, with four statistical degrees of freedom in every band. Income labels are thousands of USD per year; finite upper bounds are excluded.

In the lowest-income band, combined agreement differs by only about $4.39$ percentage points. The full distributions still differ descriptively: strong disagreement is $13.64\%$ in the male group and $27.40\%$ in the female group. The omnibus Pearson chi-squared test does not reject independence at 5%, but that decision does not prove equal distributions.

In the 10k–20k band, combined agreement is almost the same. Nevertheless, strong agreement is higher in the female group, $18.66\%$ versus $11.54\%$, while the less intense agreement category is higher in the male group, $24.36\%$ versus $16.42\%$. Collapsing the categories hides that distinction. The full-distribution p-value is approximately $0.2435$.

In the 20k–30k band, combined agreement is higher in the male group by about $12.96$ percentage points. Strong agreement alone is $27.17\%$ versus $18.30\%$. The observed difference is therefore larger than in the two lowest bands, despite its full-table p-value remaining above 5%.

In the 30k–40k band, combined agreement differs by about $17.40$ percentage points, with fewer male neutral responses, $15.69\%$ versus $26.14\%$. The full-table p-value of $0.03851$ is nominally significant at 5%. In the 40k–50k band, the combined agreement gap is a similar $16.18$ percentage points, but the full-table p-value is $0.1534$. These labels are not a direct comparison between the two effects: significant and nonsignificant results need not differ significantly.

In the highest-income band, strong agreement is $41.84\%$ in the male group and $22.15\%$ in the female group. Combined agreement differs by about $20.95$ percentage points. This band has both the largest sample size and the largest Cramér's V of the six tables, with a very small p-value. Its particularly strong evidence reflects both precision and an observed difference in the distribution.

All six minimum expected frequencies exceed five: approximately $7.15$, $9.57$, $12.02$, $15.04$, $7.93$ and $36.03$. Thus sparse expected counts do not provide a reason to reject the usual chi-squared distribution approximation here. If the six hypothesis tests form one exploratory family, a Bonferroni correction would use $0.05/6\approx 0.00833$ for each test; only the highest-income result crosses that threshold. This qualifies the unadjusted 30k–40k result without turning its observed contrast into zero.

In the elaboration model, the larger agreement gaps in the middle and upper bands provide a descriptive pattern of specification: the conditional contrast varies with income. This is not a Simpson's paradox reversal, because the combined agreement difference has the same sign in every band. Nor is the larger high-income effect size established simply by comparing its p-value with the other p-values. A formal claim of effect modification requires a stated scale and an appropriate comparison of the effects.

The pooled comparison also mixes different income compositions. The highest-income band contains $631/1042\approx 60.56\%$ of the male group but $474/1093\approx 43.37\%$ of the female group. Average each group's six agreement proportions over the same pooled income weights using standardization with common weights. This givesThe crude difference was $19.56$ percentage points, so common weighting attenuates it by about $3.30$ percentage points but leaves a substantial descriptive contrast. With average-group rather than pooled-person weights, the symmetric decomposition of a stratified mean difference instead gives an exact split into approximately $16.29$ percentage points from conditional contrasts and $3.27$ percentage points from composition. These are different explicitly defined descriptive summaries, not estimates of a causally mediated share.

An antecedent income variable could be considered in a confounding explanation, whereas an income variable on an assumed causal pathway would enter an interpretation through causal mediation. The conditional tables do not distinguish these causal structures. They also do not identify whether any adjustment introduces collider bias. The supported conclusion is income-dependent descriptive association plus a measurable composition difference, with causal interpretation left conditional on additional assumptions.

Conditioning on marital status produces five contingency tables. In each, compare the conditional row percentages. The following differences are the sample proportion with some bad experiences minus the sample proportion without them, in percentage points; negative very-safe differences mean fewer respondents feeling very safe in the former group.

The p-values test independence within the respective strata; the magnitudes and directions come from the conditional percentages and association measures. Differences in the table are calculated from the counts before rounding.

Among single respondents, the very-safe sample proportion changes from $33.95\%$ to $29.52\%$, while the unsafe sample proportion changes from $36.28\%$ to $38.10\%$. The direction resembles the weak pooled pattern, but $p=0.6114$ supplies little evidence against independence in this stratum. This is not evidence that the population association is exactly zero.

Among married respondents, the very-safe sample proportion changes from $41.18\%$ to $32.75\%$ and the fairly-safe sample proportion rises from $27.68\%$ to $35.39\%$. Unsafe remains almost unchanged, $31.14\%$ versus $31.87\%$. Thus the within-stratum association again chiefly shifts very safe toward fairly safe. Its $p=0.0257$ is below the $5\%$ significance level, while $V=0.092$ still indicates a small association.

For respondents living with a partner, very safe changes from $41.46\%$ to $28.32\%$, fairly safe from $30.08\%$ to $40.71\%$, and unsafe from $28.46\%$ to $30.97\%$. The very-safe difference is larger than in the married stratum. The printed p-value $0.04885$ would be just below the conventional $5\%$ significance level. However, this particular output is internally inconsistent: recalculating Pearson's chi-squared test from its six displayed counts gives $X^2=4.910389$, $p=0.08585$, and $V=0.14425$. The reported $V=0.144$ agrees with the counts, whereas the printed $X^2=6.900389$ would itself imply $p=0.03174$ for two statistical degrees of freedom, rather than the printed $0.04885$. Using the displayed counts, this stratum does not reject independence at $5\%$. The source PDF itself contains these incompatible figures; this is not resolved by rounding.

Among divorced respondents, very safe changes from $41.18\%$ to $37.17\%$, but unsafe also decreases, from $36.76\%$ to $30.97\%$, as fairly safe increases from $22.06\%$ to $31.86\%$. This is a redistribution toward the middle category, not a simple shift toward greater unsafety. The near-zero gamma, $-0.013$, reflects the lack of a consistent ordinal direction, while $V=0.106$ can still register distributional differences without a direction. The p-value $0.3583$ does not reject independence.

Among widowed respondents, very safe changes from $51.43\%$ to $42.47\%$, and unsafe from $14.29\%$ to $27.40\%$. The ordinal association is positive on the chosen ordering, with gamma $0.220$. Its lack of conventional statistical significance, $p=0.3158$, does not imply a smaller estimated association than in the married group: the widowed stratum has only $108$ complete observations, versus $857$ in the married stratum, and therefore less precision. All five tables have sufficiently large expected counts for the reported approximations; even the smallest expected count is about $8.10$.

The plot uses large-sample Wald confidence intervals for the difference of independent sample proportions within each stratum. They concern the individual category contrasts, whereas the tabulated Pearson's chi-squared tests compare the full three-category distributions. The intervals are unadjusted for multiple hypothesis testing; the pooled row reuses the stratum observations, so it is not a sixth independent sample.

In the elaboration model, the persistence of a generally weak positive ordinal pattern within four strata is closer to partial replication than to a complete explanation of the pooled association by marital status. The differences in estimated patterns suggest possible specification: the divorced group's shift toward the middle category differs from the widowed group's shift toward unsafe. They do not by themselves establish population effect modification. In particular, significant and nonsignificant results need not differ significantly; compare the effects using an interaction contrast or a fitted model with the appropriate interaction terms, rather than comparing their separate p-values.^[ref]

One additional omnibus check can be derived from the supplied tables without assuming a common effect. Recalculating the five statistics from the counts, including the corrected living-with-partner statistic, the joint conditional independence test for stratified contingency tables givesUnder independent-stratum sampling this tests independence within every stratum jointly; it does not test whether the association is homogeneous across strata. It does not reject at $5\%$, but this alone does not establish that stratification removed the population association: its ten-degree-of-freedom alternative is less focused than the pooled two-degree-of-freedom test, and nonrejection is not proof of equality.

The elaboration model also distinguishes an antecedent confounder from an intervening causal mediator. These cross-sectional tables do not establish the relevant time ordering or identify either role for marital status. Five separate stratum tests further require caution about multiple hypothesis testing: for example a Bonferroni correction at familywise $5\%$ would use $0.01$ per test, below all five individual p-values. That caution changes the interpretation of selected significance labels, not the observed conditional percentages.

Three defensible approaches are:

A control-limit crossing prompts investigation; it is not itself a verdict on a hospital. Wider or adjusted thresholds also reduce sensitivity, so the false-alarm objective and the cost of missed genuine problems should be made explicit.

Let $I_0$ be the indices of the true null hypotheses, let $m_0=|I_0|$, and let $V$ count false rejections. The familywise error rate isThe Bonferroni correction rejects a null exactly when its p-value is at most $\alpha /m$. The union bound givesThis controls the familywise error rate for every configuration of true and false nulls, with arbitrary dependence between their p-values. Marginal super-uniformity, $\mathbb{P}(P_i\le t)\le t$, would also suffice.

For the Holm step-down procedure, inspect the ordered p-values in ascending order and stop at the first failed comparison. If the first comparison fails, reject none; this is the convention $k=0$ when the defining set is empty. Ties can be ordered by any fixed rule.

If $m_0=0$ there can be no false rejection. Otherwise let $j$ be the rank of the first true null. Since at most $m-m_0$ false nulls precede it, $j\le m-m_0+1$ and thus $m-j+1\ge m_0$. If any true null is rejected, the step-down rule must have rejected this first true null, which requiresUsing the union bound on the true-null p-values provesThis first true null argument for Holm control also requires no independence. Holm controls the familywise error rate at level $\alpha$ under arbitrary dependence.

Now let $R$ count all rejections. The false discovery rate is the expected proportion of rejections that are false, with zero assigned when nothing is rejected:It differs from the familywise error rate: several false rejections can still represent a small proportion of a large collection of discoveries.

The BH procedure is a step-up rule. SetIf $K=0$, reject nothing; otherwise reject all hypotheses with $P_i\le \alpha K/m$. Exactly $K$ hypotheses are rejected: if more than $K$ p-values were below that threshold, the next ordered value would also satisfy its own larger threshold, contradicting maximality. Thus $R=K$. Unlike the step-down rule, a failed early comparison does not make this procedure stop.

For the proof, assume that each true-null p-value is uniform on $[0,1]$ and independent of the entire vector of the other p-values. Joint independence of all p-values is a sufficient condition; the false-null marginal distributions can be arbitrary. Mere uniformity of the true-null marginals without a dependence condition is insufficient for this argument or for general unmodified BH control.

Fix a true null $i$. Replace its p-value by zero and let $R^{(i)}$ be the number of rejections made by the Benjamini-Hochberg procedure on the modified vector. This variable depends only on the other p-values, and $R^{(i)}\ge 1$. The Benjamini-Hochberg leave-one-out identity isTo prove it, suppose $i$ is rejected with $R=r$. Decreasing its p-value to zero leaves every ordered value above rank $r$ unchanged, since it was already among the first $r$ values. Those higher ranks still fail their thresholds, while rank $r$ still succeeds; hence $R^{(i)}=r$. Conversely, suppose $R^{(i)}=r$ and $P_i\le \alpha r/m$. Restoring $P_i$ still leaves at least $r$ values at most $\alpha r/m$, so $R\ge r$. Increasing one p-value cannot increase the maximal successful rank, so $R\le R^{(i)}=r$. This gives equality and rejection of $i$.

Independence and uniformity now giveSumming over the $m_0$ true nulls proves the exact false discovery rate under independent null p-values:If the independent true-null p-values are only super-uniform, the same calculation gives the inequality instead of equality. The distinction between the two types of error control and the dependence conditions is essential: Bonferroni and Holm have the preceding guarantees without independence, while the BH proof here explicitly uses it.

The familywise error rate is the probability of rejecting at least one true null:A valid p-value is super-uniform under its null: $\mathbb{P}(p_i\le t)\le t$ for $0\le t\le 1$. The Bonferroni correction rejects $H_i$ if $p_i\le \alpha /m$. The union bound givesThis controls the familywise error rate for every configuration of true and false nulls, without any independence assumption on the p-values.

For a nonempty index set $I\subseteq \{1,\dots ,m\}$, the intersection hypothesis $H_I={\bigcap }_{i\in I}{H}_i$ asserts that every null indexed by $I$ holds. The closure of a family of statistical hypotheses consists of all these nonempty intersections, including the original hypotheses. For each intersection choose a local test ${\varphi }_I$ of size at most $\alpha$ under $H_I$. Such tests always exist: for example, within $I$ use the Bonferroni correction, rejecting $H_I$ if ${\min }_{i\in I}{p}_i\le \alpha /|I|$. More powerful local intersection tests can be substituted whenever valid.

The closed testing procedure rejects an elementary hypothesis $H_i$ only when every local test ${\varphi }_I$ with $i\in I$ rejects its intersection hypothesis. If $I_0\ne \varnothing$, the intersection $H_{I_0}$ is true. Rejecting any true $H_i$ requires rejection by its local test ${\varphi }_{I_0}$, soIf $I_0=\varnothing$, there is no possible false rejection. This proves closed-testing control of the familywise error rate. The same argument applies if intersection hypotheses themselves are reported as rejected: requiring all their supersets to reject includes $I_0$ whenever the reported intersection is true.

Finally, consider the given hierarchical family as a laminar family of sets. Call a set $I\in \mathcal{I}$ true when $I\subseteq I_0$, so its entire intersection hypothesis is true. Let $\mathcal{M}$ be the inclusion-maximal true sets in $\mathcal{I}$. Distinct members of $\mathcal{M}$ are disjoint: if they met, laminarity would make one contain the other, contradicting maximality. Consequently,Every true set $I$ lies in a maximal true $J$. If $H_I$ is rejected by the prescribed adjusted p-value rule, the maximum over its ancestors includes $J$, henceThus the event of any false rejection is contained in the union of these events over the disjoint maximal true sets. Using validity of each $p_J$ under its true intersection and the union bound,Equivalently, with probability at least $1-\alpha$ every rejection is correct. This proves familywise error control for a laminar hypothesis family. It requires neither independence nor the presence of all possible intersections or singletons in $\mathcal{I}$. Empty index sets are excluded, as required by the denominator $|J|$; if there are no true sets the result is immediate. Adjusted values exceeding one can be truncated at one without changing decisions for $\alpha <1$.

The Bonferroni correction compares every p-value with $0.05/100000=5\times 10^{-7}$. Only the first two displayed values lie below that threshold; all undisplayed values are at least the tenth displayed value. HenceFor the Benjamini-Hochberg procedure at false discovery rate level $q=0.1$, the rank-$i$ threshold is $qi/100000=i\times 10^{-6}$. The first nine displayed ranks meet their thresholds, while rank ten does not: $1.1\times 10^{-5}>10^{-5}$. Consequently nine is the intended count if every later rank also fails its threshold. The rule is a step-up rule, however:It does not stop at the first failed comparison. The full rejection count is therefore not determined by ten order statistics. For example, completing the list with $99990$ values equal to $1$ gives $k=9$, whereas completing it with $99990$ values equal to $0.02$ gives $k=100000$, since $p_{(100000)}=0.02\le 0.1$. Both completions respect the printed ten smallest p-values. Thus at least nine are rejected; exactly nine requires an extra assumption about the undisplayed ranks. In the second completion even the tenth marker is rejected, despite failing its own rank threshold. The false discovery rate guarantee also requires the usual validity and independence or appropriate positive-dependence assumptions, unlike the arbitrary-dependence Bonferroni correction.

The familywise error rate isThe Bonferroni correction rejects $H_i$ when $p_i\le \alpha /m$. Since a valid p-value is super-uniform under its null, the union bound givesNo independence assumption is needed.

For the closed testing procedure, form the intersection hypothesis $H_I={\bigcap }_{i\in I}{H}_i$ for every nonempty $I\subseteq \{1,\dots ,m\}$ and choose a level-$\alpha$ local test for each $H_I$. Reject $H_i$ exactly when every $H_I$ with $i\in I$ is rejected by its local test. If any true $H_i$ is rejected, then the intersection $H_{I_0}$ of all true nulls is rejected. Since its local test has significance level $\alpha$,This is the closed-testing control of the familywise error rate.

Write $W={\sum }_{j=1}^m{w}_j$. Procedure (A) is the Weighted Bonferroni correction: it rejects $H_i$ when $p_i\le \alpha w_i/W$. Therefore

Procedure (B) is the Weighted Holm step-down procedure. Let $k$ be the first rank in the ordering $q_{(1)}<\cdots <q_{(m)}$ whose hypothesis is a true null, and put $W_0={\sum }_{i\in I_0}{w}_i$. If any true null is rejected, the procedure reaches step $k$ andbecause every true null remains among ranks $k,\dots ,m$. Henceand another union bound gives FWER at most $\alpha$.

At step $k$, procedure (B) divides by the total weight still under consideration, which is no larger than $W$. Its critical values therefore increase as hypotheses are rejected. Moreover, if procedure (A) would reject a hypothesis, every earlier ordered $q$ also passes the initial threshold, so procedure (B) reaches and rejects it. Thus (B) contains every rejection of (A) and can make strictly more rejections while retaining the same strong FWER control.

If $B=\varnothing$, then for every $j$ at least one of $H_j^{\beta }$ and $H_j^{\eta }$ is true. Sincevalidity of the p-value for whichever component null is true implies $\mathbb{P}(q_j\le t)\le t$. Therefore the union bound givesThis is the Bonferroni correction for the composite intersection alternatives.

Four groups have $\left(\genfrac{}{}{0ex}{}{4}{2}\right)=6$ pairwise comparisons. Testing each at test size $\alpha =0.05$ without adjustment generally increases the familywise error rate, the chance of at least one type I error. If the six rejection events were independent, that chance would be $1-(1-0.05{)}^6\simeq 0.265$; shared samples make the actual comparisons dependent, so this expression is illustrative rather than exact.

A single omnibus ANOVA tests equality of all four means with one stated test size, avoiding that six-test inflation for the global question. If the aim is to identify differing pairs, follow with suitably adjusted comparisons, such as Bonferroni correction or a simultaneous pairwise method. A significant omnibus test does not automatically license six unadjusted claims. The benefit concerns control of type I error, rather than a guarantee that the model's independence and common-variance assumptions hold.