Double machine learning

Double machine learning combines Neyman orthogonality with cross-fitting to estimate a low-dimensional statistical parameter while using machine learning for nuisance functions. Orthogonality means that the expected estimating score has zero first derivative with respect to nuisance perturbations at the truth. Small nuisance errors therefore affect the target equation only at second order, under the required smoothness assumptions. cross-fitting separately limits overfitting effects by evaluating scores on observations excluded from their nuisance training.

For the augmented inverse-probability-weighted estimator, a typical sufficient condition istogether with nuisance consistency, suitable moments and treatment probabilities bounded away from zero and one. Each error being $o_p(n^{-1/4})$ is sufficient for this product condition, but one may converge more slowly if the other converges faster. Under the full regularity conditions the target estimate has asymptotic normality at scale $n^{-1/2}$, enabling confidence intervals. Machine learning prediction accuracy alone does not supply these assumptions or Causal identification.^[ref]

An estimating score has Neyman orthogonality when its population expectation has zero first derivative in every admissible nuisance direction at the truth. Writing $M(\theta ,\eta )=\mathbb{E}\psi (W;\theta ,\eta )$, this means $D_{\eta }M({\theta }_0,{\eta }_0)[h]=0$. It is a property of the score and model, not ordinary perpendicularity of observed data vectors.

For $Y={\theta }_{0}D+g_0(X)+U$, with $\mathbb{E}[U\mid X,D]=0$ and $m_0(X)=\mathbb{E}[D\mid X]$, take $\psi =(Y-\theta D-g(X))(D-m(X))$. A perturbation of $g$ contributes $-\mathbb{E}[h_g(X)(D-m_0(X))]=0$; a perturbation of $m$ contributes $-\mathbb{E}[U{h}_m(X)]=0$. Suitable second-order smoothness then makes small nuisance errors enter the population equation at higher order. Estimation still needs identification, moment and convergence-rate conditions.^[ref]