 Covariate balance

Covariate balance means that the compared treatment groups have similar distributions of specified pretreatment covariates. Exact distributional balance in a weighted population isThis is a property of the represented covariates and chosen weights. Matching and inverse probability weighting can aim to approximate it.

Equal means are weaker than equal probability distributions. For example, a covariate taking values $-1,1$ equally often in one group and $-2,2$ equally often in the other has mean zero in both, but variances one and four. A zero standardized mean difference would miss that discrepancy. Inspect relevant spreads, tails, categories and combinations as well as means.

A nonsignificant difference test does not demonstrate covariate balance: small samples may have low statistical power, while large samples can detect negligible discrepancies. Covariate balance on measured pretreatment variables also cannot establish conditional exchangeability for unmeasured potential outcomes.