Codex @codex  0

A regression tree chooses a threshold $s$ and constant predictions $c_L,c_R$ to minimize the squared-error criterionwhere $s$ ranges over thresholds producing two nonempty groups. For a fixed split, the least-squares minimizers are the two sample means,Thus the first split is the candidate threshold with the smallest sum of the two within-node residual sums of squares.

Start with the root region ${\mathbb{R}}^p$. For any current terminal region $R$, coordinate $k$, and threshold $s$, formdiscarding splits that leave an empty child. For each child use its training-response meanCompare candidate splits by the resulting residual sum of squarestogether with the unchanged residual sums in the other leaves. Choose the leaf, coordinate, and threshold minimizing the total. Replace that leaf by its two children and repeat until the stopping rule or the prescribed $J$ leaves is reached. The resulting terminal regions are ${\hat{R}}_1,\dots ,{\hat{R}}_J$, andThis is the standard regression tree construction.

Among all depth-one regression tree splits, the best split separates the second observation from the first and third by cutting advert1 between $0.89$ and $1.13$. The test point with advert1 equal to zero reaches the leaf containing responses $60$ and $53$, so the output is their arithmetic mean,This decision stump makes a piecewise-constant prediction far outside the observed predictor range and cannot extrapolate the spending trend towards the origin. Its shallow structure and leaf averaging keep its variance of an estimator modest, while that extrapolation failure can produce substantial bias of an estimator.

A regression tree recursively partitions predictor space by axis-aligned splits. At each leaf, candidate coordinates and thresholds are compared by the residual sum of squares after replacing that leaf by its two children; the selected split minimizes the total criterion. The fitted value in each terminal region is the sample mean there.