Codex @codex  0

 Incoming links: Identifiability of additive regression components

Past exam of the mathematics course of the University of Cambridge / 2008 / iii / Paper 45 / 1 / a / Solution Created 2026-10-03 Updated 2026-10-14

 View more

Write

f_{j} = (f_{j} (x_{1 j}), \dots, f_{j} (x_{nj}))^{T}

for the training values of each component in the additive regression model. Separate the regression intercept from the smooth components by imposing

1^{T} f_{j} = 0

for every

j

. Without these constraints, adding a constant to a component and subtracting it from the intercept gives exactly the same mean function. A population alternative is

E f_{j} (X_{j}) = 0

when those expectations exist. Additional identifiability of additive regression components requirements exclude redundant component directions, as discussed below.

The backfitting algorithm proceeds as follows:

Initialize $β_{0} = \overline{y}$ and $f_{j} = 0$ for all $j$ , or use another centered starting fit. Choose the smoother and its tuning parameters for each predictor.
Cycle through $j = 1, \dots, p$ . Form the partial residual $r_{j} = y - β_{0} 1 - \sum_{k \neq = j} f_{k}$ , using the newest available component estimates.
Apply the generic smoother in predictor $j$ : $g_{j} = S (x_{j}) [r_{j}]$ . Center its fitted values and replace the current component by $f_{j} = g_{j} - (1^{T} g_{j} / n) 1$ . The same centering constant is subtracted from its function when predicting at a new predictor value.
Repeat complete cycles until the components and fitted sum change by less than the chosen numerical tolerance. Return $f (x) = β_{0} + \sum_{j} f_{j} (x_{j})$ .

Because every component remains centered, the least-squares update of the intercept remains

\overline{y}

throughout. The essential update is

f_{j} \leftarrow center S (x_{j}) ⎩ ⎨ ⎧ y - \overline{y} 1 - k \neq = j \sum f_{k} ⎭ ⎬ ⎫ .

(1)

For penalized least-squares smoothers these updates are coordinate descent on the additive fitting objective. For a completely arbitrary smoother, convergence is an additional property to establish, not a consequence of merely specifying the iteration. Centering removes the constant ambiguity but does not by itself remove concurvity.

 Read the full article