A kernel support vector machine constructs a large-margin classifier in a feature Hilbert space. Given training points and labels , choose a feature map and an affine score . Prediction is the sign of the score. Normalizing the functional margin to one makes the closest separating geometric margin , and the full margin between the two supporting hyperplanes is . Maximizing that margin therefore minimizes .
For separable data, the hard-margin support vector machine primal is
Noisy or nonseparable data use slack variables and the soft-margin support vector machine:
Eliminating gives the equivalent hinge loss objective . The parameter balances margin size against violations; it is not a hard bound on their number. The bias is unpenalized here, which determines the equality constraint in the dual.
Attach multipliers to and to . The Lagrangian is
Stationarity gives , and . Eliminating yields
where is a positive-definite kernel, meaning positive semidefinite Gram matrices. The hard-margin dual is the same objective with only , without the upper bound. Soft-margin strict feasibility follows by taking and all , so strong duality and the Karush-Kuhn-Tucker conditions apply. For separable hard-margin data a separator can be rescaled to give strict margins, supplying the analogous qualification. Although the feature space may be infinite dimensional, projecting onto the finite span of the training feature vectors preserves scores and cannot increase its norm, so this optimization reduces to a finite span.
The complementary slackness relations are
Points with are support vectors. If , then and ; such a point gives . A point with can lie inside the margin or be misclassified, while contributes no term to . If no coefficient lies strictly between the bounds, an admissible bias must be obtained from the KKT inequalities, rather than dividing by a nonexistent margin vector. Bias and dual coefficients can be nonunique even when the optimal feature-space weight is unique.
The kernel trick operates both during training and prediction. The dual optimization uses only the Gram matrix , so the feature vectors need not be formed. Prediction likewise uses
A nonlinear kernel therefore makes a linear separator in feature space represent a nonlinear boundary in input space. For example, on the degree-two polynomial kernel corresponds to . Gaussian kernels yield infinite-dimensional feature spaces. An arbitrary similarity is not automatically a valid kernel: for every finite collection and real coefficients , one needs . This makes the dual quadratic form positive semidefinite and the maximization concave.
Mercer's theorem gives a spectral realization under its additional analytic hypotheses. For a continuous symmetric positive-semidefinite kernel on a compact domain with a finite full-support measure, the integral operator is compact, self-adjoint and positive. The Mercer expansion is
with the standard uniform convergence conclusions under these hypotheses. Since , the nonlinear feature map
is well defined. This explains the connection of Mercer kernels to Hilbert space features, rather than treating the kernel trick as a purely formal substitution.
The finite-Gram positivity condition is more general than this compact-domain spectral theorem. Every such kernel generates a Reproducing-kernel Hilbert space: on finite sums of kernel sections define
quotient out zero-norm elements, and complete. The resulting space satisfies the reproducing property . Its canonical feature map is , whose inner product is exactly . A feature map need not be injective; calling it an embedding does not by itself prove distinct inputs remain distinct. The kernel support vector machine uses this geometry together with convex duality to fit and evaluate a maximum-margin classifier while accessing the geometry only through kernel evaluations.
For a bounded linear operator between Hilbert spaces, let and . The Moore–Penrose inverse of an operator is defined on
For , with , define
This is independent of the chosen preimage , since two preimages differ by a vector in . Equivalently, it is the inverse of the restriction of to , applied to the component of the data in , and is zero on .
The generalized solution is the unique minimum-norm least-squares solution. Its residual is orthogonal to the range, giving the operator normal equation
The operator normal equation alone leaves an arbitrary null-space component; the second condition fixes the minimum-norm representative. The identities and hold on their appropriate domains.
For an infinite-rank compact operator, the range need not be closed, and is generally unbounded. Data outside need not have any least-squares solution at all, even though they can be approximated by range elements. This domain qualification is crucial in part (d); the Moore–Penrose notation does not turn an ill-posed inverse into an everywhere-defined bounded operator.
Use the orthonormal Fourier series basis , indexed by , and the Hilbert space inner product . The integral operator is a periodic convolution operator. Changing variables and using periodicity gives
There is no extra factor : the paper's already includes the full integral, rather than its normalized Fourier-series coefficient. The adjoint kernel is , so .
A singular system of a periodic convolution operator is therefore
Indeed and . The unit factor determined by the complex argument of in is necessary when the complex Fourier coefficients are not positive real numbers. Both families are orthonormal and complete, since every . One may enumerate by , or order the positive singular values by decreasing magnitude.
The continuous kernel on a finite square makes a Hilbert-Schmidt operator, hence compact. Since is continuously differentiable and periodic, integration by parts gives, for ,
by the Riemann-Lebesgue lemma. In particular the singular values tend to zero. Nonzero multipliers imply both and : the range is dense, but the inverse is unbounded and the range is not closed.
Let . Using the phase of in the singular value system, the Moore–Penrose inverse of an operator becomes
This gives the admissible data for a periodic convolution inverse. Because the range is dense and the kernel is zero here, the domain of the inverse is exactly the range, not all of .
For and ,
The formal expression would consequently be
However, the high-frequency obstruction for nonperiodic exponential data prevents this from being a Hilbert-space solution. The numerator is nonzero, is asymptotic to a nonzero constant divided by , and part (c) proved . Hence , so the Picard criterion fails. For , is not defined in the specified Hilbert space. The formal series is not a convergent generalized solution.
If with , the data are a single periodic Fourier mode: . Then there is an exact unique solution,
In particular, if the intended parameter is real, only is admissible, giving .
For the inadmissible cases the inverse problem still has approximate solutions : their images are Fourier projections converging to in , while their norms diverge. Thus the least-squares residual has infimum zero but no minimizer. This distinguishes an undefined exact inverse from a regularized truncated reconstruction; it is the necessary qualification to the question's unrestricted constant .
Use the genuine planar Givens rotation
In the second component the cosine multiplies : the repeated in the printed formula is an error. With that printed expression, at the pair becomes , which does not preserve length or measure. The rotation-based claims require the corrected expression. Also take , since the normalization by is undefined for .
Let and . The change of variables formula and determinant one give . Thus each is a unitary operator, with adjoint . The Kac collision operator is the average
The Minkowski integral inequality gives , so is bounded. For the Hilbert space inner product, integration and the angular change give
Hence and . In fact a nonzero radial Gaussian function is fixed by every rotation, showing . Angular averages can be understood as strong Bochner integrals; continuity of rotations in follows first for smooth compactly supported functions, then by density.
For a probability measure-preserving system, weak mixing is the vanishing of averaged absolute correlation discrepancies:
By approximation with simple functions and the Cauchy-Schwarz inequality, this is equivalent to
Here , and . Absolute values are part of the definition: signed Cesaro convergence of a sequence of these correlations alone expresses ergodicity, not weak mixing.
Suppose the product system is ergodic. Fix a mean-zero and any . On the product take
These belong to , and . The mean ergodic theorem on the product, followed by pairing with , gives
The Cauchy-Schwarz inequality in bounds the averaged absolute correlation by the square root of this quantity. Subtract the mean from a general to obtain the definition above. This proves product ergodicity implies weak mixing, through the square-correlation proof of weak mixing from product ergodicity.
For the multiple averages, write and use the following Van der Corput lemma. If is bounded in a Hilbert space, every correlation average
exists, and , then in norm. One way to see the estimate is to replace by ; for fixed the change in its long average tends to zero. The Cauchy-Schwarz inequality and expansion of the squared block norm give
Let . This proves the auxiliary implication needed here.
A weak mixing system is ergodic: a mean-zero invariant would have the nonvanishing correlation . Every positive power is also weak mixing, since for nonnegative correlation discrepancies ,
This part of the stability of weak mixing under powers and products will control the induction.
In fact the stronger arithmetic-progression multiple averages under weak mixing holds:
For this is the mean ergodic theorem and ergodicity. Suppose it is known for and first assume . Put . For fixed , set
Using invariance of the integral to remove the common gives
This identity does not require an inverse of . Apply the induction hypothesis to the factors and pair their limit with . It follows that
The average in of the right side tends to zero because is weak mixing and . The Van der Corput lemma proves the zero limit. For general , split it into and its constant mean; the first term has the zero limit just proved, and the other term is times the induction average for . This completes the induction.
Pair this result with the real bounded . For
the requested Cesaro limit is
To obtain convergence in density of a sequence, we also need the product system to be weak mixing. For simple tensors, its correlations are products of single-system correlations. If and in averaged absolute discrepancy and both sequences are bounded, then
so the product discrepancy has zero average. Finite sums of tensors are dense in , and the Cauchy-Schwarz inequality extends the conclusion to arbitrary functions. Hence is weak mixing.
Apply the multiple-average result to on that product. Because the original are real,
Together with the first-moment limit this gives
The mean-square criterion for convergence in density now yields, for every ,
Therefore the density convergence of multiple weak-mixing correlations gives the same answer:
The second-moment argument is essential; the signed Cesaro limit alone would not imply this conclusion.
Use the corrected essential spectrum of a bounded self-adjoint operator and put . Essential spectral points are real. If is infinite-dimensional, choose an orthonormal sequence in the kernel. It converges weakly to zero by the Bessel inequality, and its residuals vanish.
If is finite-dimensional, membership in the essential spectrum means the range is not closed. Choose unit with , using the closed-range bound on the kernel complement. A bounded Hilbert space sequence has a weakly convergent subsequence. Its weak limit satisfies because bounded operators preserve weak convergence, and ; hence . This subsequence is a singular Weyl sequence.
Conversely a singular Weyl sequence first places in the spectrum of a bounded operator. If it were not essential, the sequential properness for a self-adjoint operator equivalence for would yield a norm-convergent subsequence. Its weak limit is zero, whereas norm convergence of unit vectors gives a unit norm limit, a contradiction. Therefore
For nonreal , the resolvent lower bound excludes such a sequence, so the equivalence covers all .
Throughout this question the Banach spaces and Hilbert spaces are complex. Separability is an additional standing hypothesis, not part of the general definition of a Hilbert space. Here and below unit ball means the closed unit ball. An open unit ball in positive finite dimension is not compact. In finite dimension, equivalence of norms identifies the closed unit ball with a closed bounded subset of Euclidean space, so the Heine-Borel theorem gives compactness in the norm topology.
In an infinite-dimensional Hilbert space, the Gram-Schmidt process constructs an infinite orthonormal sequence . Put . Then and, for , . No subsequence is a Cauchy sequence, contradicting sequential compactness. Thus the closed unit ball is norm compact exactly in finite dimension.
Use the assumed separability of the Hilbert space to choose a countable Hilbertian basis . For , every scalar sequence is bounded. Repeated extraction followed by the diagonal subsequence argument gives one subsequence with all these coordinates converging, say to . The Bessel inequality gives for every , so exists and .
For , let be its finite basis truncation. Coordinate convergence gives , while the Cauchy-Schwarz inequality bounds the remaining inner product by , uniformly in . First choose large, then large. Thus
This proves the asserted sequential conclusion. To obtain actual compactness for the weak topology, note that on a bounded ball it is induced by the metric
Finite coordinates and the uniformly small tail show that gives coordinate convergence; uniform boundedness of the ball extends this to every inner product as above. Hence this metric induces exactly the restricted weak topology. In a metric space, sequential compactness implies compactness. The closed unit ball is therefore weakly compact, and the limit remains inside it.
In the state-picture convention, a Stinespring representation of a completely positive map consists of an auxiliary Hilbert space and a linear map such that
The partial trace discards the environment. For example, from a Kraus representation , take . The adjoint, or observable-picture, form is .
A general completely positive map does not require to be an linear isometry of Hilbert spaces. If is trace preserving, then , so is an linear isometry of Hilbert spaces. This is the Stinespring dilation of a quantum channel, which will be used in the data-processing proof.
Choose an orthonormal basis of the joint -dimensional Hilbert space whose first vector is the given purification of a density operator . Apply rank-one dephasing to in this basis, and denote the resulting diagonal probabilities by . Their first entry is
The preceding entropy increase under nonselective projective measurement and entropy bound with one prescribed probability yield
Therefore the quantum Fano inequality is
The quantity is the entanglement fidelity of on ; it is already a squared overlap, so it is , rather than , that enters the binary entropy. The argument is an instance of the entropy bound from overlap with a pure state in dimension . At the output is the original pure state and its Von Neumann entropy is zero. For the system is trivial and the same zero-entropy conclusion holds without evaluating .
For the quadratic variational principle for a symmetric positive operator below, use a symmetric operator on the real Hilbert space . Strict positive definiteness means
A uniformly positive definite symmetric operator satisfies the stronger coercive operator condition
In this variational setting, “positive definite” is often used for a symmetric operator with this uniform bound. We will state explicitly where the coercive operator bound is needed. For a bounded linear operator defined on all of , symmetry means that the operator is self-adjoint. For an unbounded operator, positivity is imposed on its operator domain, and the variational formulation is made on its form domain.
Symmetry is essential in a real Hilbert space: positivity of the quadratic expression alone does not imply symmetry. For example, with nonzero real skew-symmetric matrix satisfies , but its quadratic functional has derivative involving , not . A positive definite symmetric operator supplies both the positivity and symmetry needed in part (b).
For a densely defined operator on a complex Hilbert space, unbounded self-adjointness means equality including equality of the operator domains. Symmetry alone asserts only and is insufficient. Use an inner product linear in its first argument. If , symmetry gives
Thus is bounded below. Since is closed, its range is closed; its orthogonal complement is , so the range is also dense and therefore all of . The inverse is bounded by . The nonreal resolvent estimate for a self-adjoint operator proves .
For bounded self-adjoint , its numerical range of an operator and numerical radius are
Every number in is real. If , its distance from that closed set is positive, and . The same bound applies to , so the range is dense as well as closed and has a bounded inverse. Hence the numerical-range spectral enclosure is
The closure is necessary in infinite dimension: the endpoints of the numerical range need not themselves be attained.
Put and , a bounded positive semidefinite operator. Choose unit vectors with . The Cauchy-Schwarz inequality for the positive form implies
Therefore is an approximate eigenvalue. Apply the same argument to , where , to get the other endpoint. This proves the numerical-range endpoint approximate-eigenvalue theorem. Each endpoint is in the spectrum: a bounded inverse would forbid unit vectors with vanishing residual.
If an unbounded self-adjoint operator is positive semidefinite, then for every real , . The range argument above again proves invertibility, excluding negative values from the spectrum. For the converse, suppose . Its resolvent operator is bounded and self-adjoint. The resolvent spectral mapping identity puts : for , the relevant spectral parameter of is , and nonreal values are excluded by self-adjointness. By the endpoint result just proved, . Positive-form Cauchy-Schwarz inequality then gives
For this yields . Thus the spectral positivity criterion for a self-adjoint operator is
This resolvent proof uses the bounded numerical-range result to handle the full unbounded domain.
Use the Holstein–Primakoff transformation on the physical occupation number subspace . With the canonical commutation relation , the ordered square root gives
Both endpoints are respected: and . Consequently,
Since these Fock states form a basis of the physical spin space, there. Similarly . The Holstein–Primakoff occupation constraint is essential: unrestricted bosonic occupation would not represent a spin- Hilbert space.
Set and , so . By the chain rule, the equation becomes . The substitution reduces it further to
Multiplication by and integration by parts gives for a nonzero solution, ruling out zero and negative eigenvalues. For , the first boundary condition gives , and the second gives , . Thus all eigenpairs are
with arbitrary nonzero scalar multiples of each eigenfunction. These are the complete list since all solutions of the constant-coefficient differential equation have been considered; the transformed sine system is also complete in the corresponding weighted Hilbert space.
Work with real-valued functions. The screened sine-Gordon energy is well defined: , and is integrable by the Cauchy-Schwarz inequality. It is coercive, because
A minimizing sequence is bounded in the Hilbert space . Weak sequential compactness of bounded sequences in a reflexive Banach space supplies a weakly convergent subsequence. The squared H1 space norm is weakly lower semicontinuous, the source pairing is weakly continuous, and the nonlinear term is covered by local Sobolev compactness gives lower semicontinuity of a nonnegative integral. The direct method in the calculus of variations therefore gives a minimizer .
Taking its first variation in any gives
Thus the Euler-Lagrange equation is
as a weak solution, equivalently in distributions when tested against smooth compactly supported functions. The derivative of the nonlinear term is justified by and the second-order remainder bound .
Now , so the supplied elliptic regularity estimate puts in . Applying the Sobolev inequality to and each first weak derivative gives . Morrey's inequality and uniformly continuous integrable functions vanish at infinity prove that its continuous representative tends to zero.
The final printed supremum estimate is false in general. The maximum bound for a monotone reaction term involves , which is odd and strictly increasing: , and its zeros are isolated. If , a positive maximum of satisfies ; apply the same argument to . The valid general estimate is
The bound by is valid if , but can be smaller than for larger positive .
For an explicit failure of the source-size bound for the screened sine-Gordon equation, take , , and . These are smooth functions in the required spaces. With ,
Hence . Moreover has , so the energy is convex and this critical point is a minimizer. The example therefore satisfies even the minimizing and hypotheses of the printed claim.