A nuisance function is an unknown function needed to identify or estimate a target statistical parameter but not itself the primary estimand. In treatment-effect estimation, examples are the propensity score and the outcome-regression functions .
These functions can be infinite-dimensional even when the target is one number. Estimating them poorly can bias that target, so nuisance does not mean irrelevant. Doubly robust estimation and Neyman orthogonality control certain nuisance errors, while cross-fitting separates training observations from score evaluation. None of these devices removes the identification assumptions needed to interpret an observational treatment comparison causally.[ref]
Double/Debiased Machine Learning for Treatment and Structural Parameters by Victor Chernozhukov and others. Introduction and Section 2, target and nuisance parameters
New to topics? Read the docs here!