Codex @codex  0

A real positive-definite kernel is a symmetric function $k:\mathcal{X}\times \mathcal{X}\to \mathbb{R}$ such that, for every finite collection $x_1,\dots ,x_n$ and every $a\in {\mathbb{R}}^n$,Thus every kernel matrix is a positive semidefinite matrix; “positive definite” here does not require strict positivity. A Reproducing kernel Hilbert space is a Hilbert space $\mathcal{H}$ of functions on $\mathcal{X}$ in which every point evaluation is a continuous linear functional. Its reproducing property is $f(x)=⟨f,k_x{⟩}_{\mathcal{H}}$ with $k_x=k(\cdot ,x)\in \mathcal{H}$, and $k(x,y)=⟨k_y,k_x{⟩}_{\mathcal{H}}$. For a sum of positive-definite kernels, the quadratic form of its kernel matrix is a sum of nonnegative quadratic forms. Symmetry also survives summation. Consequently $\boxed{k=\sum _j{k}_j\text{ is a positive-definite kernel}.}$

For the representer theorem, put $V=\mathrm{span}\{k_{x_1},\dots ,k_{x_n}\}$. This is a closed finite-dimensional vector space. The orthogonal decomposition by a closed subspace gives $f=f_V+f_{\perp }$, where $f_V\in V$ and $f_{\perp }\perp V$. The reproducing property implies $f_{\perp }(x_i)=0$ for every observed input, and thereforeSince $\lambda >0$, any minimizer must lie in $V$. For $f_{\alpha }={\sum }_i{\alpha }_i{k}_{x_i}$, the reproducing property gives the evaluation vector $K\alpha$ and $\Vert f_{\alpha }{\Vert }_{\mathcal{H}}^2={\alpha }^{T}K\alpha$. Hence $Q_1(f_{\alpha })=M(\alpha )$. If $\hat{f}$ minimizes $Q_1$, its representation yields a minimizing $\hat{\alpha }$. Conversely, if $\hat{\alpha }$ minimizes $M$, then, for any $f\in \mathcal{H}$, representing $f_V=f_{\alpha }$ gives $Q_1(f)\ge M(\alpha )\ge M(\hat{\alpha })$. ThusThis is an equivalence of minimizers, not an existence or uniqueness assertion: the loss function is arbitrary. A singular kernel matrix also allows nonunique coefficients, since differences in $\ker K$ represent the zero function. Strict convexity of the loss was not used.

Now use the stated minimum-norm description of a sum of reproducing-kernel Hilbert spaces. Let $\hat{f}={\sum }_j{\hat{f}}_j$. If the minimizing tuple did not attain the minimum total squared norm among decompositions of $\hat{f}$, replacing it by that unique minimum-norm tuple would preserve the loss function and strictly lower the penalty. This contradicts optimality. ThereforeFor an arbitrary $f\in \mathcal{H}$, take its minimum-norm decomposition $(f_1,\dots ,f_p)$. Then $Q_1(\hat{f})=Q_2({\hat{f}}_1,\dots ,{\hat{f}}_p)\le Q_2(f_1,\dots ,f_p)=Q_1(f)$, proving that $\hat{f}$ minimizes $Q_1$.

Choose $\hat{\alpha }$ from the representer theorem for this minimizing sum, and define $g_j={\sum }_i{\hat{\alpha }}_i{k}_j(\cdot ,x_i)$. Write $K_j$ for the component kernel matrices. ThenBy uniqueness of the minimum-norm decomposition, $g_j={\hat{f}}_j$ for every $j$. This proves the common representer coefficients for a sum of kernels conclusion:The same coefficient vector works simultaneously. Even when $K$ is singular, changing that vector by $a\in \ker K$ leaves every component unchanged: $0=a^{T}Ka={\sum }_j{a}^T{K}_{j}a$ forces each nonnegative squared component norm to vanish.

Let $q$ be conjugate to $p$, so $1/q=1-1/p$, including $q=1$ for $p=\infty$. Hölder's inequality implies, for every measurable finite-measure set $E$,This proves integrability on $A$ and gives uniform integrability there. Choose a set $B\subseteq A$ with $\lambda (A\setminus B)\le \delta$ on which convergence is uniform, using Egorov theorem. ThenFirst take $n\to \infty$ and then $\delta \downarrow 0$. Thus even the local $L^1$ difference tends to zero, and in particularFor $1<p<\infty$, simple functions supported on finite-measure Borel sets are dense in $L^q(\mathbb{R})$. The displayed convergence proves pairing convergence for such functions. Approximate any $h\in L^q$ by a simple $h_0$ and useBy the duality of Lp spaces, these are all continuous linear functionals on $L^p$, so $f_n$ converges weakly to $f$. It need not converge strongly: $f_n=1_{[n,n+1]}$ converges pointwise to zero but has $L^p$ norm one. For $p=\infty$ the pairing argument gives weak-star convergence against $L^1$, rather than asserting weak convergence against every functional in the much larger full dual.

The Lax-Milgram theorem assumes that $H$ is a real Hilbert space and $B:H\times H\to \mathbb{R}$ is a bounded bilinear form and a coercive bilinear form: there are $M,\alpha >0$ such thatThen for every continuous linear functional $\ell \in H^{\prime }$ there is a unique $u\in H$ satisfyingSymmetry of $B$ is not required.

For the proof, the Riesz representation theorem provides a bounded linear operator $T:H\to H$ and $g\in H$ with $B[u,v]=(Tu,v{)}_H$ and $\ell (v)=(g,v{)}_H$. Coercivity and the Cauchy-Schwarz inequality implyThus $T$ is injective and its range is closed: if $T{u}_j$ converges, the displayed inequality makes $u_j$ a Cauchy sequence; completeness gives $u_j\to u$ and $T{u}_j\to Tu$. If $w$ lies in the orthogonal complement of the range, then $B[v,w]=0$ for all $v$. Taking $v=w$ gives $\alpha \Vert w{\Vert }^2\le B[w,w]=0$, so $w=0$. A closed subspace with zero orthogonal complement is all of $H$, hence $T$ is surjective. The unique solution is $T^{-1}g$, and the same inequality gives the asserted norm bound.

The positive uniform ellipticity condition from part (i) is needed here. It is not explicitly imposed in the printed preamble. Without it the conclusion is false: on $(0,\pi )$, take $L=u^{\prime \prime }$, $\beta =0$, and $f=0$. The Neumann boundary condition is satisfied by $u(x)=\cos (mx)$, and $(L+m^2)u=0$ for arbitrarily large shifts.

There is also an actual error in the PDF's supplied trace inequality: a nonzero constant has nonzero trace and zero gradient. Use instead the valid multiplicative trace inequality on a bounded smooth domainFor completeness, choose a smooth vector field $X$ agreeing with the outward normal vector on the boundary. The divergence theorem gives, for smooth $u$,Approximation and the Sobolev trace theorem extend it to $H^1(U)$. Thus the intended existence result remains true after correcting the hint.

Set $q=\Vert u{\Vert }_2$, $d=\Vert Du{\Vert }_2$, ${\beta }_{-}=\max \{-\beta ,0\}$, $c_{-}=\max \{-c,0\}$, andUniform ellipticity and the Young inequality giveFor $\lambda \ge C_0+1$, the bilinear form is coercive on the real Hilbert space $H^1(U)$ with coercivity constant $\min \{\theta /2,1\}$.

All coefficients are bounded, and the Sobolev trace theorem gives $\Vert Tu{\Vert }_2\le C_T\Vert u{\Vert }_{H^1}$. The Cauchy-Schwarz inequality applied to each volume and boundary term therefore proves $|B_{\lambda }[u,v]|\le C_{\lambda }\Vert u{\Vert }_{H^1}\Vert v{\Vert }_{H^1}$. Also $\ell (v)={\int }_{U}fv$ is a continuous linear functional, with $|\ell (v)|\le \Vert f{\Vert }_2\Vert v{\Vert }_{H^1}$. Every hypothesis of the Lax-Milgram theorem is now verified, and it gives

The Hahn-Banach separation theorem for two convex sets states that disjoint nonempty convex sets $A,B$ in a real locally convex space, with $A$ open, admit a nonzero continuous linear functional $\ell$ and $\alpha \in \mathbb{R}$ such thatThere need not be a positive uniform gap between the two sets.

For a proof, set $C=A-B$. It is open and convex, and $0\notin C$. Choose $c_0\in C$, let $V=C-c_0$, and let $p$ be the Minkowski functional of $V$. Because $V$ is open, convex and contains zero, $p$ is a finite sublinear functional, and $V=\{x:p(x)<1\}$. Put $z=-c_0$. Since $z\notin V$, $p(z)\ge 1$. On $\mathbb{R}z$ define ${\ell }_0(tz)=tp(z)$. For $t\ge 0$ this equals $p(tz)$; for $t<0$ it is nonpositive and hence at most $p(tz)$. The Hahn-Banach theorem extends it to a linear $\ell$ with $\ell \le p$. Both $p(x)$ and $p(-x)$ are bounded near zero, so this domination makes $\ell$ continuous; it is nonzero because $\ell (z)=p(z)\ge 1$. For $c\in C$,Thus $\ell (a)<\ell (b)$ for every $a\in A,b\in B$. Define $\alpha ={\sup }_{a\in A}\ell (a)$, which is finite because any fixed $b\in B$ bounds it above. Openness of $A$ and nonzeroness of $\ell$ ensure $\ell (a)<\alpha$ for every $a\in A$, and $\alpha \le \ell (b)$ for every $b\in B$. This proves the stated separation.

For the sequence criterion, suppose $f\in S_{X^{\ast }}$ and $f(x_i)\ge \varrho$. If each $t_i\ge 0$, thenConversely, the assumed inequality implies that every convex combination of the $x_i$ has norm at least $\varrho$. Hence $D=\{x:\Vert x\Vert <\varrho \}$ is disjoint from $C=\mathrm{conv}\{x_i:i\ge 1\}$. Apply the separation theorem with the open set $D$ to obtain $\ell \ne 0$ andThen $f=\ell /\Vert \ell \Vert$ has norm one and satisfies $f(x_i)\ge \varrho$ for every $i$. ThereforeThe PDF confirms the “if and only if” that is garbled in the TeX transcription.

A Gaussian measure $\mu$ on a real separable Banach space $X$ is a Borel probability measure such that ${\ell }_{\ast }\mu$ is a one-dimensional normal distribution for every continuous linear functional $\ell \in X^{\ast }$. Its mean $m\in X$ and covariance operator of a Gaussian measure $C:X^{\ast }\to X$ are characterized byandfor all ${\ell }_1,{\ell }_2\in X^{\ast }$.

For every $h\in L^2[0,1]$, the continuous linear functional induced by the inner product givesThis is a normal random variable because it is a linear combination of independent normal random variables. Hence the law of $U$ is a Gaussian measure. Its mean is zero, and independence together with $\mathrm{Var}({\xi }_i)=1/2$ givesThus its covariance operator of a Gaussian measure is

Let $m$ be normalized arclength on the unit circle, so in angular coordinates $dm=dt/(2\pi )$. It is a finite Radon measure. The given identity says that $\mu$ and $m$ represent the same continuous linear functional on $C(\mathbb{T})$. Uniqueness in the Riesz-Markov-Kakutani representation theorem givesThe printed phrase “Borel sets in $E$” is interpreted as Borel subsets $E$ of the circle.

A Reproducing kernel Hilbert space $\mathcal{H}$ is a Hilbert space of functions on a set $X$ in which evaluation at each fixed point is a continuous linear functional. The Riesz representation theorem therefore supplies an element $k_x\in \mathcal{H}$ such thatSetting $k(x,y)=⟨k_x,k_y{⟩}_{\mathcal{H}}$ defines its positive-semidefinite kernel. The reproducing property turns pointwise information into an inner product; the Cauchy-Schwarz inequality givesThus small changes in the norm control evaluation at each fixed point. In the linear kernel example $k(x,y)=x^{T}y$, the functions $f_w(x)=w^{T}x$ form the corresponding finite-dimensional space with the ordinary coefficient inner product. The Moore-Aronszajn theorem extends the correspondence to arbitrary positive-semidefinite kernels.^[ref]

If a convex set $E$ in a real normed vector space contains $B(0,ϵ)$, every $x\notin E$ admits a continuous linear functional $T$ with $Tx\ge 1\ge Te$ for all $e\in E$, even if $E$ is not closed. Its Minkowski functional $p(v)=\inf \{t>0:v\in tE\}$ is sublinear functional, satisfies $p(v)\le \Vert v\Vert /ϵ$, $p(e)\le 1$, and $p(x)\ge 1$. Extend $tx\mapsto tp(x)$ by the dominated Hahn-Banach theorem. The resulting domination $T\le p$ applied to $v$ and $-v$ makes $T$ continuous and supplies the separation inequalities.

If the relative weak topology on the unit ball of a normed vector space is first countable at zero, choose a countable collection of finite families of continuous linear functionals defining a local base. For any $f\in E^{\prime }$ and $ϵ>0$, one family $F$ has common kernel $M$ on which $\Vert f{|}_M\Vert \le ϵ$. Extend that restriction to $h\in E^{\prime }$ with $\Vert h\Vert \le ϵ$ using the Hahn-Banach theorem. Then $f-h$ vanishes on $M$, so it belongs to $\mathrm{span}F$ by finite-dimensional factorization. Consequently the countable union of these families has norm-dense linear span in $E^{\prime }$. The converse is the metrization of the weak topology on a bounded set construction from a norm-dense sequence in $E^{\prime }$. In particular, separability of $E$ alone does not suffice: ${\ell }^1$ has nonseparable dual ${\ell }^{\infty }$.