Sum of reproducing-kernel Hilbert spaces (${\mathcal{H}}_{\sum k_j}$)

For positive-semidefinite kernels $k_j$ with Reproducing-kernel Hilbert spaces ${\mathcal{H}}_j$, the Reproducing-kernel Hilbert space of $k={\sum }_j{k}_j$ consists of sums $f={\sum }_j{f}_j$, withThe minimum is attained uniquely. Indeed, in the Hilbert space direct sum $E={⨁}_j{\mathcal{H}}_j$, the tuples summing to zero form a closed subspace $N$: they are the intersection, over inputs $x$, of the kernels of the continuous maps $(f_j)\mapsto {\sum }_j{f}_j(x)$. Every affine fibre has a unique representative in $N^{\perp }$ by orthogonal decomposition by a closed subspace. The vector $(k_j(\cdot ,x){)}_j$ lies in $N^{\perp }$ and represents evaluation in the quotient; its evaluation at $y$ is ${\sum }_j{k}_j(y,x)$, identifying the positive-semidefinite kernel with $k$.

For a real-valued loss function depending only on evaluations at $x_1,\dots ,x_n$, and a positive squared-norm penalty, any minimizing tuple in a sum of reproducing-kernel Hilbert spaces has the formwith the same coefficient vector for every component. The tuple must be the unique minimum-norm decomposition of its sum. Apply the representer theorem to that sum and note that the displayed tuple has total squared norm ${\sum }_j{\hat{\alpha }}^T{K}_j\hat{\alpha }={\hat{\alpha }}^T({\sum }_j{K}_j)\hat{\alpha }$, exactly the norm of the sum. This argument is conditional on existence of a minimizer; arbitrary loss functions need not attain their infimum.