= Cross-fitting
<Cross-fitting> separates nuisance-function training from the observations used to evaluate an estimating equation. Divide <independent random variables>[independent] observations into $K$ disjoint groups $I_1,\ldots,I_K$. For each group $I_k$, fit <nuisance function>[nuisance functions] $\widehat\eta^{(-k)}$ using all observations outside $I_k$, then evaluate the score $\psi(W_i;\theta,\widehat\eta^{(-k)})$ only for $i\in I_k$. Combine the held-out scores, for example by solving
$$
\frac1n\sum_{k=1}^K\sum_{i\in I_k}\psi(W_i;\theta,\widehat\eta^{(-k)})=0.
$$
Conditional on the training data for a particular group, its held-out <independent random variables>[independent] observations do not influence the fitted <nuisance function>[nuisance functions]. This reduces overfitting terms in the analysis while allowing every observation to contribute to the final estimate. <cross-validation> usually selects a tuning parameter by held-out prediction performance; <cross-fitting> instead constructs an <estimator> or estimating equation. The group-specific fits overlap, so their resulting estimates are not generally <independent random variables>[independent]. Dependent data require a splitting scheme and theory adapted to that dependence.<Double Debiased Machine Learning for Treatment and Structural Parameters by Victor Chernozhukov and others>[arXiv v6, Section 3, Definitions 3.1–3.2, K-fold DML1 and DML2 constructions]{ref}
Back to article page
