The one-nearest-neighbour classification boundary consists of the portions of the Voronoi diagram separating cells whose observed labels differ. For these nine grid points it forms diagonal and vertical or horizontal perpendicular-bisector segments around the three triangular observations. Every training point is its own nearest neighbour, so, absent a distance tie convention that excludes the query itself, its training error is zero.
An unpruned maximal CART classifier repeatedly cuts with vertical or horizontal lines until every terminal rectangle is pure or contains observations that cannot be separated by an axis-aligned split. Here the distinct grid points can be isolated into pure rectangles, producing a step-shaped, axis-aligned decision boundary and zero training error.
The random forest boundary is the majority vote of many bootstrap-grown axis-aligned trees. It remains piecewise axis-aligned but averages away many unstable individual cuts, so one should sketch a less extreme boundary enclosing regions supported repeatedly by the triangular points. Its resubstitution training error is typically small and can be zero, but bootstrap omission and voting mean that zero is not guaranteed. The relevant built-in estimate is instead the out-of-bag error, which the output gives as ; this large value reflects the tiny sample and unstable labels near the class boundary.
Articles by others on the same topic
There are currently no matching articles.