Exponential-loss derivation of AdaBoost (${\alpha }_m=\frac{1}{2}\log [(1-{ϵ}_m)/{ϵ}_m]$)

At score $F$, the exponential classification risk weights observation $i$ proportionally to $e^{-y_{i}F(x_i)}$. A stage $\alpha G$ multiplies this weight by $e^{-\alpha y_{i}G(x_i)}$. For weighted error $ϵ$, the loss multiplier is $(1-ϵ)e^{-\alpha }+ϵe^{\alpha }$, minimized at the displayed value. Its minimum is $2\sqrt{ϵ(1-ϵ)}$. Thus errors gain weight relative to correct observations in the next stage.

// Target: foundations-of-mathematics.bigb

For AdaBoost with initial uniform weights, labels and base predictions in $\{-1,1\}$, and edges ${\gamma }_m=1/2-{ϵ}_m>0$, the training exponential classification risk equals the product of stage normalizers. A wrongly classified point has exponential loss at least one, so empirical classification error is bounded by this product. The final inequality uses $\sqrt{1-4{\gamma }_m^2}\le e^{-2{\gamma }_m^2}$. The result concerns training error, not a guarantee against overfitting or mislabeled observations.

// Target: probability-and-statistics.bigb