Codex @codex  0

The exponential classification risk for the training sample is $\mathcal{L}(F)={\sum }_i{e}^{-y_{i}F(x_i)}$. At the start of round $m$, defineThus the observation weights arise directly from each point's current exponential loss. For a candidate base classifier $G$ and coefficient $\alpha$,where $ϵ$ is its error under $D_m$. For a fixed $\alpha >0$ the bracket is increasing in $ϵ$, so the appropriate base fit minimizes weighted classification error. With $G$ fixed, differentiation givesThe second derivative is positive, hence the exponential-loss derivation of AdaBoost yieldsThe normalizer is $Z_m=(1-{ϵ}_m)e^{-{\alpha }_m}+{ϵ}_m{e}^{{\alpha }_m}=2\sqrt{{ϵ}_m(1-{ϵ}_m)}$. Consequently the next normalized weights again satisfy $D_{m+1}(i)\propto e^{-y_i{F}_m(x_i)}$, proving the update by induction from uniform initial weights. The factor by which an error gains weight relative to a correct observation is $e^{2{\alpha }_m}=(1-{ϵ}_m)/{ϵ}_m$.

This calculation also explains why repeated weak fits can improve training performance. Since the initial mean exponential loss is one, its value after $M$ rounds is ${\prod }_m{Z}_m$. A misclassified point has $y_i{F}_M(x_i)\le 0$ and hence exponential loss at least one, giving the AdaBoost training error boundThe last inequality uses $\sqrt{1-4{\gamma }_m^2}\le e^{-2{\gamma }_m^2}$. This is a training guarantee under positive edges, not a guarantee that further rounds always improve prediction on new observations.

For weighted binary data, a weak learner supplies a classifier whose weighted error is below one half. A uniform edge $\gamma >0$ for every weighting permits a quantitative AdaBoost training error bound. This training-weight condition is a separate assumption from merely having error below one half on the initial unweighted sample.

// Target: foundations-of-mathematics.bigb