Codex @codex  0

Confounding occurs when the compared treatment groups differ in relevant potential outcomes, so their observed outcome contrast mixes the causal effect with differences in baseline prognosis. In a binary-treatment model, absence of confounding is expressed through ignorability, $Y(a)\perp A$ for each compared $a$, or through conditional exchangeability after a sufficient pretreatment adjustment set is specified.

For a causal directed acyclic graph, $A\leftarrow U\to Y$ is the simplest backdoor path: the common cause changes both assignment and outcome. Confounding by indication is a practical example when baseline severity influences which treatment is received. Outcome standardization, matching and inverse probability weighting can address measured confounding under their identification assumptions.

A changed adjusted estimate does not, by itself, prove that confounding was removed. Noncollapsibility of the odds ratio can produce different marginal and conditional odds ratios even without confounding; adjustment for an inappropriate variable can introduce collider bias. Causal structure and the target effect must guide the adjustment choice.^[ref]

Covariate balance means that the compared treatment groups have similar distributions of specified pretreatment covariates. Exact distributional balance in a weighted population isThis is a property of the represented covariates and chosen weights. Matching and inverse probability weighting can aim to approximate it.

Equal means are weaker than equal probability distributions. For example, a covariate taking values $-1,1$ equally often in one group and $-2,2$ equally often in the other has mean zero in both, but variances one and four. A zero standardized mean difference would miss that discrepancy. Inspect relevant spreads, tails, categories and combinations as well as means.

A nonsignificant difference test does not demonstrate covariate balance: small samples may have low statistical power, while large samples can detect negligible discrepancies. Covariate balance on measured pretreatment variables also cannot establish conditional exchangeability for unmeasured potential outcomes.

Detection-threshold selection bias in pulse-energy means occurs when detected pulses have a different energy distribution from all emitted pulses. A detector conditions the reported mean on detection, not merely on emission. Let $U$ be emitted-pulse energy with finite mean and let $s(u)=\mathrm{Pr}(D\mid U=u,E)$ be detection probability conditional on emission $E$. ThenThe denominator must be positive. An ideal energy threshold has $s(u)=1$ above threshold and zero below. Its nondecreasing selection preferentially retains stronger pulses; false triggers or selection on another variable require a different model.

For an original distribution, emitted energies are $1$, $3$ and $7\mathrm{nJ}$ with probabilities $0.25$, $0.50$ and $0.25$. Their conditional pulse-energy mean given emission is $3.5\mathrm{nJ}$. A perfect receiver accepting only energies above $2\mathrm{nJ}$ detects $75\%$ of emitted pulses and reportsThe detected-event mean is about $23.8\%$ too high for the emitted-pulse mean, even with perfect energy measurements on every accepted event.

A trigger counter, a lower-threshold reference monitor and a threshold sweep can distinguish missing detections from missing emissions. Historical calorimetric inference uses total energy, exposure time and pulse rate; the rate must describe the population whose mean is requested.^[ref] This selection example is an independent probability model, not a reconstruction of that experiment.

Known positive detection probabilities can support inverse probability weighting. A hard threshold with zero probability below it leaves that part of the energy distribution unidentified from detected events alone. Reporting the acceptance fraction as well as the detected-event mean makes the conditioning explicit; it does not recover the unseen weak-pulse energies.

The Horvitz-Thompson estimator estimates a finite population total by expanding each observed unit's contribution by the reciprocal of its inclusion probability. D. G. Horvitz and D. J. Thompson introduced the general unequal-probability construction in 1952.^[ref]

Let $y_1,\dots ,y_N$ be fixed values attached to labelled population units. A sampling design selects a random subset $S$, with known inclusion probabilities ${\pi }_i=\mathbb{P}(i\in S)>0$. Thenwhere $I_i$ is the inclusion indicator. This is inverse probability weighting for a finite population.^[ref] Its expected value is immediatelyThe subscript $p$ emphasizes design-based inference: the fixed values are not assumed to follow a probability model; the sample-selection mechanism supplies the randomness. The inclusion indicators need not be independent. Dividing by known $N$ gives an unbiased estimator of the finite population mean.

For an original exact example, use values $(y_1,y_2,y_3)=(2,4,6)$ and choose samples $\{1,2\}$, $\{1,3\}$, $\{2,3\}$ with probabilities $1/2,1/3,1/6$, respectively. The inclusion probabilities are $(5/6,2/3,1/2)$. The corresponding estimates of the total are $8.4,14.4,18$, andSimply multiplying each sample mean by three instead gives totals $9,12,15$, with expected value $11$. Equal treatment of the observed values ignores the unequal chances of observing them.

The sampling variance depends on pair information. Put ${\pi }_{ij}=\mathbb{P}(i,j\in S)$, including ${\pi }_{ii}={\pi }_i$. Bilinearity of covariance givesFor the three-unit design it is $14.4$, found directly from $\frac{1}{2}(8.4-12{)}^2+\frac{1}{3}(14.4-12{)}^2+\frac{1}{6}(18-12{)}^2$. Thus an unbiased estimator need not give a nearly correct total in each sample.

With pairwise inclusion positivity, the same expansion device applied to pairs constructs the Horvitz-Thompson variance estimator:Its expected value equals the preceding population sum because ${\mathbb{E}}_p(I_i{I}_j/{\pi }_{ij})=1$. In the worked design its three possible values are $9.76,58.56,-60$, whose probability-weighted average is $14.4$. This particular unbiased estimator of variance can therefore be negative for an individual sample. Its expected value identity is a statement about repeated selection.

If ${\pi }_i=0$, an arbitrary unobserved $y_i$ cannot contribute to an unbiased total estimate. If a distinct pair has ${\pi }_{ij}=0$, its product cannot be recovered by the pair-expansion argument. Finally, dividing the weighted sum by the random sum of its inverse-probability weights changes the estimator into a ratio; the indicator proof above no longer establishes that this ratio is an unbiased estimator. The known population size and the actual sampling design are part of the result.

Inverse probability weighting weights observations by reciprocals of the probabilities that generated their observed assignment or selection. For binary treatment with propensity score $e(X)$, the average-treatment-effect observation weight isIt uses the probability of the treatment actually received. Under conditional exchangeability, consistency in causal inference and positivity, the identification identity isTo prove it, condition on $X$: the assignment probability cancels its denominator, leaving $\mathbb{E}(Y\mid A=a,X)$. Average over $X$ and apply the causal identification assumptions. The same cancellation with a function $g(X)$ instead of $Y$ proves population balance of the measured covariates under correctly specified assignment probabilities.

If a stratum has treatment probability $0.2$, each treated observation there receives weight five: it represents units with the same measured baseline characteristics who usually receive the other treatment. Very small assignment probabilities give large weights and unstable estimates. Estimated assignment models, trimming and normalization can change finite-sample behavior and, for trimming, the target population. Overlap weights deliberately define a different target rather than estimating the same full-population contrast by another name.

James M. Robins develops statistical methods for causal inference in epidemiology. His joint exposition with Miguel A. Hernán explains how inverse probability weighting can estimate population treatment effects from observational studies under appropriate identifying assumptions.^[ref]

Weighting gives more influence to people whose observed treatment was relatively unlikely given their measured baseline covariates. In the resulting weighted comparison, measured treatment-selection differences are removed under the assumed model. Validity still depends on conditional exchangeability, consistency of potential outcomes and positivity in causal inference; extreme weights also expose a practical difficulty when some treatment choices are rare.^[ref]

Missing data are intended measurements that were not observed. Write $R=1$ when an outcome $Y$ is observed and let $X$ denote fully observed covariates. The observation mechanism matters because observed cases can differ systematically from the target population.

Missing completely at random means $R$ is independent of both $Y$ and $X$. Missing at random allows observation to depend on $X$, but requires $R$ and $Y$ to be conditionally independent given $X$. Missing not at random permits additional dependence on the unobserved $Y$. For example, follow-up determined only by recorded age can be missing at random after conditioning on age; follow-up depending on unrecorded current symptoms can be missing not at random. The missing-at-random assumption generally cannot be established from the observed data alone.

An observed-case analysis need not estimate the population mean even under missing at random, because the distribution of $X$ among observed cases can be different. Regression, Inverse probability weighting or Multiple imputation can address this under suitable assumptions, including a positive probability of observation throughout the target population. Stating those assumptions is part of the statistical model rather than a consequence of an imputation algorithm.

Conditional on $X$,Multiplying by $1\{X_1=x_1\}$, taking expectations, dividing by $\mathbb{P}(X_1=x_1)$, and using part (i) proves the inverse probability weighting formula.