Nested cross-validation (source code)

= Nested cross-validation

Nested cross-validation separates selection from assessment using two levels of held-out data. Within each outer training fold, inner <cross-validation> chooses a model or tuning parameters; the complete selected fitting procedure is then applied to that outer training fold and scored on its untouched outer evaluation fold. This estimates the performance of the <model selection> procedure, including its choices, rather than reusing its best inner score as an assessment. A final model can then be selected and fitted using all available <training data>. https://jmlr.org/papers/v11/cawley10a.html[Cawley and Talbot, selection bias and performance evaluation]{ref}