Past exam of the mathematics course of the University of Cambridge 2008 iii Paper 45 1 c Solution Created 2026-10-03 Updated 2026-10-14
To isolate the bias-variance tradeoff of the smoothing step, fix the three neighbor locations and write , with independent zero-mean errors of common variance . At a target , the local estimate isThe bias and variance of a three-neighbour weighted smoother areConsequently the variance is minimized by equal weights, , when it equals . It decreases as increases from zero to , and increases thereafter toward as . In particular, variance is not increasing over the entire interval .
Giving more weight to the closest location reduces the influence of more distant responses and often reduces smoothing bias. If the closest location is the target itself, , put . Then the squared bias is , andThus a locally flat signal favors equal averaging, while stronger local smoothing bias favors greater nearest-point weight. At a general target the bias is affine in , and its absolute value need not decrease: farther neighbors can cancel one another's approximation errors. Even without that cancellation, a smoothness-based worst-case bias bound decreases with because the first neighbor is closest. The exact tradeoff is therefore lower variance near balanced weights versus potentially lower bias near the closest-point fit. These formulas describe the elementary smoother; fitted partial residuals, centering and repeated backfitting can introduce dependence, so the variance of the final additive fit requires its full fitting operator rather than this single-step calculation.