 Cross-fitting

Cross-fitting separates nuisance-function training from the observations used to evaluate an estimating equation. Divide independent observations into $K$ disjoint groups $I_1,\dots ,I_K$. For each group $I_k$, fit nuisance functions ${\hat{\eta }}^{(-k)}$ using all observations outside $I_k$, then evaluate the score $\psi (W_i;\theta ,{\hat{\eta }}^{(-k)})$ only for $i\in I_k$. Combine the held-out scores, for example by solvingConditional on the training data for a particular group, its held-out independent observations do not influence the fitted nuisance functions. This reduces overfitting terms in the analysis while allowing every observation to contribute to the final estimate. cross-validation usually selects a tuning parameter by held-out prediction performance; cross-fitting instead constructs an estimator or estimating equation. The group-specific fits overlap, so their resulting estimates are not generally independent. Dependent data require a splitting scheme and theory adapted to that dependence.^[ref]