Doubly robust estimation

Doubly robust estimation combines two nuisance functions so that either one can be correctly estimated while the other is misspecified. For example, let $A\in \{0,1\}$ indicate treatment, $X$ covariates, $m_a(X)=\mathbb{E}[Y\mid A=a,X]$, and $e_a(X)=\mathbb{P}(A=a\mid X)$. Under conditional exchangeability, positivity and the usual consistency of observed and potential outcomes, ${\mu }_a=\mathbb{E}[m_a(X)]$ is the mean potential outcome under treatment $a$.

For candidate functions ${\tilde{m}}_a,{\tilde{e}}_a$ with denominators bounded away from zero, the augmented signal isConditioning on $X$ shows thatThe bias vanishes if either nuisance function is correct. This algebra explains Double robustness and the second-order product bias. It does not protect against both models being wrong, failed Causal identification, or nearly zero denominators. cross-fitting helps when flexible functions are estimated from the same sample; valid confidence intervals require further rate and moment assumptions. The average treatment effect is ${\mu }_1-{\mu }_0$.^[ref]

A nuisance function is an unknown function needed to identify or estimate a target statistical parameter but not itself the primary estimand. In treatment-effect estimation, examples are the propensity score $e(x)=\mathbb{P}(A=1\mid X=x)$ and the outcome-regression functions $m_a(x)=\mathbb{E}[Y\mid A=a,X=x]$.

These functions can be infinite-dimensional even when the target is one number. Estimating them poorly can bias that target, so nuisance does not mean irrelevant. Doubly robust estimation and Neyman orthogonality control certain nuisance errors, while cross-fitting separates training observations from score evaluation. None of these devices removes the identification assumptions needed to interpret an observational treatment comparison causally.^[ref]

An augmented inverse-probability-weighted estimator averages a regression prediction plus an inverse-probability-weighted residual. In the treatment setting of doubly robust estimation, let ${\hat{m}}_a(X_i)$ estimate the conditional outcome mean and ${\hat{e}}_a(X_i)$ the probability of receiving treatment $a$. ThenThe first term predicts outcomes for everyone; the weighted residual corrects prediction errors using observed outcomes from the corresponding treatment group. The conditional-expectation calculation in doubly robust estimation makes the remaining bias a product of outcome-regression and propensity-score errors. An exactly correct outcome regression makes the residual correction mean zero even with an incorrect propensity score; a correct propensity score makes that correction repair the regression bias.

The nuisance estimates are commonly obtained by cross-fitting, with each observation evaluated using functions trained on other observations. Both treatment probabilities must remain positive on the population being studied, and small fitted probabilities can make the variance very large. Double robustness concerns consistency under one correct nuisance model; it does not automatically deliver valid finite-sample or asymptotic confidence intervals.

An estimator is doubly robust when it remains consistent if either of two nuisance models is correctly specified, commonly an outcome-regression model or a propensity-score model.

Cross-fitting separates nuisance-function training from the observations used to evaluate an estimating equation. Divide independent observations into $K$ disjoint groups $I_1,\dots ,I_K$. For each group $I_k$, fit nuisance functions ${\hat{\eta }}^{(-k)}$ using all observations outside $I_k$, then evaluate the score $\psi (W_i;\theta ,{\hat{\eta }}^{(-k)})$ only for $i\in I_k$. Combine the held-out scores, for example by solvingConditional on the training data for a particular group, its held-out independent observations do not influence the fitted nuisance functions. This reduces overfitting terms in the analysis while allowing every observation to contribute to the final estimate. cross-validation usually selects a tuning parameter by held-out prediction performance; cross-fitting instead constructs an estimator or estimating equation. The group-specific fits overlap, so their resulting estimates are not generally independent. Dependent data require a splitting scheme and theory adapted to that dependence.^[ref]

Double machine learning combines Neyman orthogonality with cross-fitting to estimate a low-dimensional statistical parameter while using machine learning for nuisance functions. Orthogonality means that the expected estimating score has zero first derivative with respect to nuisance perturbations at the truth. Small nuisance errors therefore affect the target equation only at second order, under the required smoothness assumptions. cross-fitting separately limits overfitting effects by evaluating scores on observations excluded from their nuisance training.

For the augmented inverse-probability-weighted estimator, a typical sufficient condition istogether with nuisance consistency, suitable moments and treatment probabilities bounded away from zero and one. Each error being $o_p(n^{-1/4})$ is sufficient for this product condition, but one may converge more slowly if the other converges faster. Under the full regularity conditions the target estimate has asymptotic normality at scale $n^{-1/2}$, enabling confidence intervals. Machine learning prediction accuracy alone does not supply these assumptions or Causal identification.^[ref]

An estimating score has Neyman orthogonality when its population expectation has zero first derivative in every admissible nuisance direction at the truth. Writing $M(\theta ,\eta )=\mathbb{E}\psi (W;\theta ,\eta )$, this means $D_{\eta }M({\theta }_0,{\eta }_0)[h]=0$. It is a property of the score and model, not ordinary perpendicularity of observed data vectors.

For $Y={\theta }_{0}D+g_0(X)+U$, with $\mathbb{E}[U\mid X,D]=0$ and $m_0(X)=\mathbb{E}[D\mid X]$, take $\psi =(Y-\theta D-g(X))(D-m(X))$. A perturbation of $g$ contributes $-\mathbb{E}[h_g(X)(D-m_0(X))]=0$; a perturbation of $m$ contributes $-\mathbb{E}[U{h}_m(X)]=0$. Suitable second-order smoothness then makes small nuisance errors enter the population equation at higher order. Estimation still needs identification, moment and convergence-rate conditions.^[ref]