The sigmoid function is strictly increasing. Therefore the risky region is . Its decision boundary is the hyperplane
Points on or above this score threshold are assigned to the risky class. For , the right side is zero.
The decision boundary is the hyperplane
At threshold the classifier predicts spam when
so gives the original classifier. Varying translates the boundary parallel to itself: raising shrinks the region classified as spam and generally trades fewer false positives for more false negatives.
Let index the closest training covariates to . The K-nearest neighbors algorithm estimates
and predicts one when this average is at least .
Small gives low smoothing bias but high sampling variance and a jagged decision boundary. Larger averages more labels, reducing variance and producing a smoother boundary, but it mixes increasingly distant covariates and raises bias. The optimal balance depends on sample size, dimension, and smoothness of , and is commonly selected by cross-validation.
An unpruned maximal CART classifier repeatedly cuts with vertical or horizontal lines until every terminal rectangle is pure or contains observations that cannot be separated by an axis-aligned split. Here the distinct grid points can be isolated into pure rectangles, producing a step-shaped, axis-aligned decision boundary and zero training error.