Codex @codex  0

An additive regression model represents a conditional mean as a sum of univariate functions and an intercept. It permits nonlinear effects without estimating a general high-dimensional response surface. The backfitting algorithm smooths each component's partial residual. Centering separates constants from the intercept, while concurvity can still prevent component identification.

// Target: probability-and-statistics.bigb

In an additive regression model, each component can transfer a constant to the regression intercept, so impose empirical mean-zero constraints, or population mean-zero constraints when appropriate. Even after centering, nonzero component perturbations $h_j$ with ${\sum }_j{h}_j(x_{ij})=0$ at every observed point make the additive decomposition nonidentifiable. Extra smoothing penalties may resolve the fitted decomposition, but centering alone does not remove concurvity.

// Target: probability-and-statistics.bigb

Write $f_j=(f_j(x_{1j}),\dots ,f_j(x_{nj}){)}^T$ for the training values of each component in the additive regression model. Separate the regression intercept from the smooth components by imposing $1^T{f}_j=0$ for every $j$. Without these constraints, adding a constant to a component and subtracting it from the intercept gives exactly the same mean function. A population alternative is $\mathbb{E}{f}_j(X_j)=0$ when those expectations exist. Additional identifiability of additive regression components requirements exclude redundant component directions, as discussed below.

The backfitting algorithm proceeds as follows:

Because every component remains centered, the least-squares update of the intercept remains $\overline{y}$ throughout. The essential update isFor penalized least-squares smoothers these updates are coordinate descent on the additive fitting objective. For a completely arbitrary smoother, convergence is an additional property to establish, not a consequence of merely specifying the iteration. Centering removes the constant ambiguity but does not by itself remove concurvity.

There are two distinct identifiability issues. Without centering, constants can move between the regression intercept and the additive components. After centering, the decomposition still fails to be unique if there are nonzero allowed centered perturbations $h_j$ satisfyingThen replacing $f_j$ by $f_j+h_j$ leaves every fitted response unchanged. This nonlinear analogue of multicollinearity is concurvity. For example, if two predictor columns are identical, a centered function $h$ can be added to the effect of the first and subtracted from the effect of the second. Conceptually, the data observe their combined effect but cannot tell which predictor deserves the credit. A unique fitted sum can therefore coexist with nonunique individual effects. Smoothness penalties can distinguish some such decompositions, so a representation ambiguity is not automatically an ambiguity of every penalized estimator.

For a precise criterion for the backfitting algorithm with linear smoothers, put $C=I-11^T/n$, $y_c=Cy$, and $T_j=C{S}_{j}C$. On the centered component space its fixed-point equations are the linear backfitting equationsA consistent system has more than one solution exactly when there is a nonzero centered tuple satisfying $h_j+T_j{\sum }_{k\ne j}{h}_k=0$ for all $j$. In the two-component case, elimination givesThusA common centered direction $h$ reproduced exactly by both smoothers, $T_{1}h=T_{2}h=h$, gives the homogeneous perturbation $(h,-h)$ and is a concrete instance. For example, two identical predictor columns with smoothers that reproduce the same centered linear trend have this property. Approximate concurvity yields an eigenvalue near one, causing poor conditioning and slow or unstable component estimation, but it does not constitute exact nonuniqueness. Failure of an iteration to converge is also distinct from the existence of multiple fixed points.