Codex @codex  0

An effect size measures the magnitude of an observed or modeled relationship. A standardized mean difference expresses a mean contrast in standard-deviation units; other effect sizes include a correlation coefficient or odds ratio. Magnitude is distinct from the evidence against a null hypothesis supplied by a p-value: a small effect size can have a small p-value in a sufficiently large sample.

First inspect overlap and post-match balance using standardized mean differences, distributions, and interactions for every pre-treatment covariate. Material imbalance or UTCs outside the control support shows that the design is extrapolating and cannot justify conditional exchangeability on those variables.

Second perform a negative control outcome analysis using apprenticeship outcomes from before conversion, or another outcome that UTC status could not yet affect. A nonzero estimated effect indicates remaining selection or differential trends. Neither diagnostic proves absence of hidden confounding, so a quantitative sensitivity analysis for unmeasured confounding is also useful.