Codex @codex  0

An affine linear classifier assigns binary labels according to the sign of an affine function. Its boundary is an affine hyperplane, and its positive region is a closed half-space under the displayed convention. On ${\mathbb{R}}^d$ the full class has VC dimension $d+1$: arbitrary signs interpolate on $d+1$ affinely independent points, while the intersecting convex hull partition theorem prevents shattering any $d+2$ points. Requiring $b=0$ gives the homogeneous case and a different VC dimension.

An affine linear classifier in ${\mathbb{R}}^k$ has VC dimension $k+1$. The upper bound follows from the intersecting convex hull partition theorem: for any $k+2$ points, label the two parts oppositely. The two convex hulls cannot be strictly separated by an affine hyperplane, since they intersect. A weak boundary convention gives the same contradiction: an affine score is nonnegative on the positive convex hull and strictly negative on the negative convex hull. The lower bound comes from the $k+1$ affinely independent vertices of a simplex, because an affine score can interpolate arbitrary signs there.

Consequently, parts (i) and (ii) immediately giveThe one-sided orientation of the lifted separators permits the sharper, exact conclusionTo prove the remaining upper bound, take any $d+2$ distinct points and a nonzero affine dependence ${\lambda }_i$ as in part (c). Write $Q={\sum }_i{\lambda }_i\Vert x_i{\Vert }^2$. Reverse every coefficient if necessary so that $Q\ge 0$. Prescribe positive labels when ${\lambda }_i>0$ and negative labels when ${\lambda }_i<0$; zero coefficients may be labelled arbitrarily. If a Euclidean ball realized these labels, its score $g(x)=r^2-\Vert x-c{\Vert }^2$ would satisfyIndeed, every positive-coefficient term is nonnegative and every negative-coefficient term is strictly positive, with at least one of the latter. On the other hand, expanding the score and using the two affine dependence identities givesa contradiction. No $d+2$ points are shattered, completing the exact determination of the VC dimension of Euclidean balls. In the exceptional zero-dimensional Euclidean space, every positive-radius Euclidean ball is the whole singleton space, so the VC dimension is zero.

Expanding the squared Euclidean norm givesIn lifted coordinates $(z,t)=\Phi (x)=(x,\Vert x{\Vert }^2)$, this is the closed half-spaceEvery labelling of the original points obtainable by a Euclidean ball is therefore obtainable on their lifted images by an affine linear classifier. The feature map is injective, since it preserves the first $d$ coordinates; distinct original points remain distinct. If the original points are shattered, every labelling of the lifted points is realized, as required.

Here “linear separator” means a separator with an intercept, namely an affine hyperplane. If a convention demands strict separation on the finite sample, increase the constant term of the positive score $r^2-\Vert c{\Vert }^2+2c\cdot z-t$ by a sufficiently small positive amount. Positive points then have strictly positive score, and the finitely many negative scores remain strictly negative. A homogeneous separator can alternatively be obtained after adjoining one more constant coordinate; this distinction matters for dimension counts.

A real positive-semidefinite kernel on an input set $X$ is a function $K:X\times X\to \mathbb{R}$ for which there is an inner product space $\mathcal{F}$ and a feature map $\varphi :X\to \mathcal{F}$ satisfyingEquivalently, $K$ is symmetric and every finite Gram matrix $(K(x_i,x_j){)}_{i,j}$ is positive semidefinite: ${\sum }_{i,j}{a}_i{a}_{j}K(x_i,x_j)\ge 0$ for all real coefficients $a_i$. An infinite-dimensional Hilbert space is allowed.

Replace every inner product $x_i\cdot x_j$ in part (b) by $K(x_i,x_j)$ and solveThis is precisely the Lagrangian dual problem obtained by replacing each original input by $\varphi (x_i)$ in the primal problem. The weight vector it determines isand the intercept follows from the same Karush-Kuhn-Tucker conditions. For example, if ${\alpha }_i>0$,Thus training needs only the finite Gram matrix, even if the feature map has infinitely many coordinates. This is the kernel trick. The result is an affine linear classifier in the feature space; its boundary in the original input space can be nonlinear. The squared-slack diagonal term and the equality constraint remain unchanged.

Every set of at least $d+2$ points in ${\mathbb{R}}^d$ can be partitioned into two nonempty parts with intersecting convex hulls. To see this, take a nonzero affine dependence on $d+2$ of its points. Its positive and negative coefficient sums have the same magnitude $L>0$. Dividing by $L$ expresses one point as a convex combination of the positive-coefficient points and also as a convex combination of the negative-coefficient points. Distribute zero-coefficient and remaining points arbitrarily between the parts. This theorem obstructs arbitrary labels for affine linear classifiers.

The positive-radius closed Euclidean balls in ${\mathbb{R}}^d$ have VC dimension $d+1$ for $d\ge 1$. Vertices of a regular simplex are shattered: interpolate any desired signs by an affine function $a\cdot x+b$, then perturb a sphere through the vertices to the score $1+\eta b+\eta a\cdot x-\Vert x{\Vert }^2$. For the upper bound, take an affine dependence $\lambda$ among $d+2$ points and orient it so that $Q={\sum }_i{\lambda }_i\Vert x_i{\Vert }^2\ge 0$. Labelling the positive coefficients inside and the negative coefficients outside would force ${\sum }_i{\lambda }_i(r^2-\Vert x_i-c{\Vert }^2)>0$, while the affine dependence makes this sum equal $-Q\le 0$. The weaker $d+2$ upper bound from unrestricted lifted affine linear classifiers loses their fixed quadratic coefficient.