For one-column Gram matrices, unit Euclidean norm of the columns gives . The restricted isometry constant formula therefore gives . For a two-column Gram matrix, put . The difference from the identity matrix has the form
up to the convention for the complex inner product. Its characteristic polynomial is , so the eigenvalues are and its matrix 2-norm is . Maximizing over pairs gives , where is the mutual coherence.
Part (b), together with the minimality defining the restricted isometry constant, gives . Every summand defining cumulative coherence is at most the mutual coherence, so . Thus, for normalized columns and ,
The upper range matters because the printed definition of cumulative coherence stops at . For , only is needed; the maximum over pairs defining mutual coherence is otherwise empty.
Write and . The injectivity of makes this Gram matrix a positive-definite matrix, since for . Thus its matrix inverse exists. Construct the least-norm dual certificate
On the active coordinates, . For , symmetry of the real Gram matrix and its matrix inverse gives
Condition (iii) makes the absolute value of this coordinate strictly less than one. Consequently is a strict dual certificate for basis pursuit, and part (c) applies. If is empty, take ; the zero vector uniquely minimizes the L1 norm on its feasible set. Condition (iii) supplies an explicit certificate and therefore unique recovery. There is no claim that this particular least-norm dual certificate is necessary: other valid certificates may exist when this one fails.
Let replicate , so . The positive-definite Gram matrix is invertible. Multiplication by and taking expected values give
All quantities are integrable: the finite-atom representation from completeness gives finite terminal values, and the matrix has finite entries. Positive definiteness also guarantees uniqueness of holdings, since would imply and hence .
Thus the replication cost is
Consequently the one-period Gram-matrix replication formula is
The symmetry of makes the displayed orientations consistent. Positivity of this would require an additional no-arbitrage condition; positive definiteness of the payoff Gram matrix alone proves the representation and uniqueness, not positivity of prices across states.
The ordinary column Gram matrix is . Use the normalized empirical Gram matrix
so that standard Gaussian entries give . This is the normalization needed for concentration around .
The restricted isometry property of order with constant means that the normalized map approximately preserves the Euclidean norm of all vectors with at most nonzero coordinates:
Equivalently, every principal block with satisfies . The least such is its restricted isometry constant. In the unnormalized definition apply this property to itself.
For a standard normal variable , direct Gaussian integration gives for . Independence therefore yields the moment-generating function of a chi-squared distribution, centred here at its mean:
The inequality follows from . For , the Chernoff bound with gives
At the trivial probability bound suffices. This proves the requested bound, with a stronger prefactor one.
For the lower tail, gives for . Taking yields . Combining both tails gives the useful chi-squared concentration inequality
For set . A direct calculation shows
Thus the exponent is at least , proving
The same threshold bounds the two-sided tail.
Finally fix a deterministic and put . If , independence of the Gaussian rows gives independently. Therefore
The two-sided bound just proved supplies
For the quadratic form is deterministically zero; the strict inequality makes the formula valid in that case too. When , replacing the threshold by gives an absolute bound independent of . In particular every fixed such direction concentrates at rate for a fixed confidence level. The restricted isometry property requires a simultaneous statement over sparse directions; this fixed-direction calculation alone is not that stronger assertion.
A kernel support vector machine constructs a large-margin classifier in a feature Hilbert space. Given training points and labels , choose a feature map and an affine score . Prediction is the sign of the score. Normalizing the functional margin to one makes the closest separating geometric margin , and the full margin between the two supporting hyperplanes is . Maximizing that margin therefore minimizes .
For separable data, the hard-margin support vector machine primal is
Noisy or nonseparable data use slack variables and the soft-margin support vector machine:
Eliminating gives the equivalent hinge loss objective . The parameter balances margin size against violations; it is not a hard bound on their number. The bias is unpenalized here, which determines the equality constraint in the dual.
Attach multipliers to and to . The Lagrangian is
Stationarity gives , and . Eliminating yields
where is a positive-definite kernel, meaning positive semidefinite Gram matrices. The hard-margin dual is the same objective with only , without the upper bound. Soft-margin strict feasibility follows by taking and all , so strong duality and the Karush-Kuhn-Tucker conditions apply. For separable hard-margin data a separator can be rescaled to give strict margins, supplying the analogous qualification. Although the feature space may be infinite dimensional, projecting onto the finite span of the training feature vectors preserves scores and cannot increase its norm, so this optimization reduces to a finite span.
The complementary slackness relations are
Points with are support vectors. If , then and ; such a point gives . A point with can lie inside the margin or be misclassified, while contributes no term to . If no coefficient lies strictly between the bounds, an admissible bias must be obtained from the KKT inequalities, rather than dividing by a nonexistent margin vector. Bias and dual coefficients can be nonunique even when the optimal feature-space weight is unique.
The kernel trick operates both during training and prediction. The dual optimization uses only the Gram matrix , so the feature vectors need not be formed. Prediction likewise uses
A nonlinear kernel therefore makes a linear separator in feature space represent a nonlinear boundary in input space. For example, on the degree-two polynomial kernel corresponds to . Gaussian kernels yield infinite-dimensional feature spaces. An arbitrary similarity is not automatically a valid kernel: for every finite collection and real coefficients , one needs . This makes the dual quadratic form positive semidefinite and the maximization concave.
Mercer's theorem gives a spectral realization under its additional analytic hypotheses. For a continuous symmetric positive-semidefinite kernel on a compact domain with a finite full-support measure, the integral operator is compact, self-adjoint and positive. The Mercer expansion is
with the standard uniform convergence conclusions under these hypotheses. Since , the nonlinear feature map
is well defined. This explains the connection of Mercer kernels to Hilbert space features, rather than treating the kernel trick as a purely formal substitution.
The finite-Gram positivity condition is more general than this compact-domain spectral theorem. Every such kernel generates a Reproducing-kernel Hilbert space: on finite sums of kernel sections define
quotient out zero-norm elements, and complete. The resulting space satisfies the reproducing property . Its canonical feature map is , whose inner product is exactly . A feature map need not be injective; calling it an embedding does not by itself prove distinct inputs remain distinct. The kernel support vector machine uses this geometry together with convex duality to fit and evaluate a maximum-margin classifier while accessing the geometry only through kernel evaluations.
The Frankl-Wilson theorem. Let be a prime number, let have elements, and let . Suppose that for every , whereas for all distinct members. Then
with terms beyond understood as zero.
Work over the finite field . For each member define its intersection polynomial
Replace every positive power by to obtain a multilinear polynomial of polynomial degree at most . This Boolean multilinearization preserves its values at every characteristic vector of a set. At the characteristic vector of a set ,
It is zero if and nonzero if . Evaluating a proposed linear relation at each therefore establishes linear independence of the . The vector space of multilinear polynomials of polynomial degree at most has the basis of monomials , , of size . This proves the Frankl-Wilson theorem by the polynomial method in combinatorics. The empty product for is , and the argument still applies.
The constant-intersection family bound, directly. Write and assume . The cases are immediate. If , the nonempty members are pairwise disjoint and there can also be the empty set, giving
For , every member has size at least . If every member has size strictly greater than , form the real matrix whose rows are their characteristic vectors of sets. Its Gram matrix satisfies
For any nonzero real vector ,
Thus the rows are linearly independent and . If a member has size , it lies inside every other member. The sets , , are nonempty and pairwise disjoint, so . This also gives
These bounds are sharp: the singletons together with attain at , while all complements of singletons attain at when .
The constant t-wise intersection dichotomy. In its substantive form the last argument requires . The printed statement does not explicitly impose this. Without it the -fold condition can be vacuous and the claimed dichotomy is false: take , , , , and . There is no -tuple to test, no common -set, and , so . We therefore prove the intended assertion for and record this necessary qualification.
If , the first alternative always holds with . Suppose . Choose indices attaining the minimum intersection size , and write
For each of the remaining indices put . The hypothesis gives for distinct remaining indices. This reduces the constant t-wise intersection dichotomy to a constant-intersection family bound on the -element set . The two possibilities are treated below.
With the centred data, no intercept is needed. Use the Lasso normalization
Write and . The KKT conditions are
Indeed, the subdifferential of the L1 norm consists of vectors with at nonzero coordinates and at zero coordinates. The subgradient optimality condition is . convexity makes these conditions sufficient as well as necessary. These are the Karush-Kuhn-Tucker conditions for the Lasso.
Here is a correlation-parameter version of least angle regression. Put , a Gram matrix, and initialize , , .
  1. Compute . For , keep . If , stop: all regression residual correlations already vanish. Otherwise add the uniquely maximizing variable to obtain .
  2. At step , put , , , and . These active signs have entries in . Solve , set , and follow the segment
  3. For each inactive variable set . Along the segment its correlation is , where . Its candidate hitting distances are
    Discard undefined or nonpositive candidates. Choose the smallest remaining distance below , if one exists, and otherwise choose . Set . If , add the unique variable achieving the hit to form and repeat; if , stop.
The two hitting formulas come from . This is LAR hitting time. Active variables are not dropped if a coefficient crosses zero: that is the distinction between this algorithm and the modified Lasso path algorithm.
The active Gram matrix is an invertible matrix under the assumed unique positive hits. Inductively, a candidate column in the span of the current active columns would have equal to a fixed linear combination of their correlations, hence proportional to on the whole segment. Its ratio could not first reach one at a positive interior knot. Thus every uniquely entering column adds a new independent direction. This also allows more predictors than observations; the algorithm stops before a dependent direction needs to be added at zero correlation.
We now prove the requested invariant. At the first positive knot the newly active correlation has magnitude . At a later knot the old active correlations have that knot's magnitude by induction, and the entering correlation has the same magnitude by the hitting rule. Throughout the next segment,
The first-hitting rule also ensures for every inactive . If no positive hit remains, this inequality continues to zero, where all correlations vanish. Consequently, on every printed closed segment,
This is the LAR active correlation invariant.
If active nonzero coefficients have the same signs as their residual correlations, the active equalities are precisely the nonzero-coordinate Lasso KKT conditions. An active coefficient which is zero also satisfies the zero-coordinate KKT conditions, because its correlation is within . Inactive coefficients are zero and satisfy those same inequalities by construction. Thus every point of the least angle regression path is a Lasso minimizer for . Under the stated uniqueness assumption,
This proves the desired implication, in fact under the weaker sign compatibility of LAR and Lasso condition requiring agreement only at nonzero coefficients.
There is an endpoint convention in the printed sufficient condition. With the standard , a newly entering coefficient is zero at a positive knot although its correlation is ; the displayed sign equality therefore cannot literally hold there. At the final zero knot the correlation signs are zero as well. Interpret sign compatibility on positive open segments, or in the nonzero-coefficient/subgradient sense just proved. The KKT conditions already handle zero coefficients at knots, and no equality of solution paths at is needed for the requested conclusion.
The ordinary least squares objective has gradient . The normal equation therefore gives
The Gram matrix is a positive-definite matrix because has full column rank. More explicitly, for any ,
since . This proves that the displayed solution is the unique global minimum.
The fitted values are , where the hat matrix is
The inverse of the symmetric matrix is symmetric, so . Multiplication gives . Thus is the orthogonal projection onto the column space of the design matrix.
The regression residual vector is with . Consequently , , and the residual sum of squares is
By fitted-residual orthogonality, . Using , the cross-covariance matrix is
Both vectors are jointly normal, so they are also independent by independence of uncorrelated jointly normal variables. The zero covariance calculation itself only needs the common error variance and lack of error correlations.
For the first wind fit, let be wind velocity and electrical output. The simple linear regression is , with independent . It has two fitted mean parameters and residual degrees of freedom. The missing analysis of variance entries are therefore velocity degrees of freedom , velocity mean square , and residual degrees of freedom . The residual mean square is , and , consistent with the printed value after rounding.
Because the model includes a regression intercept, the coefficient of determination is the explained sum of squares divided by the total centered sum of squares:
A large coefficient of determination does not rule out a wrong mean function. In the PDF's first residual-versus-fitted plot, residuals are negative at both ends and positive in the middle. This curved pattern agrees with the visibly flattening output-versus-velocity relationship and motivates a polynomial regression with a quadratic term.
The second wind model is , again with independent common-variance normal errors. Its fitted mean is . The line marked (A) is a two-sided Student t-test of against , conditional on retaining the intercept and linear term. Under the null hypothesis,
Its two-sided p-value is . This is far below , so reject a purely linear mean in favour of the quadratic fit. Equivalently, the extra-term nested-model F-test has and null law .
The third fit is a reciprocal-predictor regression, , with estimated mean . It has two mean parameters rather than the quadratic model's three. Its residual standard error is smaller, versus , and its coefficient of determination is larger, versus . The corresponding residual sums of squares are approximately and . The reciprocal fit is preferable on both these fit measures and parsimony. The two models are not nested, so an ordinary extra-term F-test between them is inappropriate. On the common normal-error likelihood, the difference also favours the third model.
Neither lower residual sum of squares nor a higher coefficient of determination establishes adequate assumptions. The quadratic residual-versus-fitted plot removes the original pronounced curvature; the reciprocal plot also has no comparably obvious mean trend. The low-output residuals appear somewhat more spread out, so check scale-location plots and residuals against velocity for heteroscedasticity. Q-Q plots assess normality; residuals against observation order assess serial dependence; regression leverage and Cook's distance identify influential observations. Further cross-validation, replicate observations at comparable velocities, and prediction errors would help choose between their extrapolation behaviours. Both fitted shapes should be judged principally over the observed positive-velocity range.
Put . If minimizes distance, then for every and every real ,
Both signs of arbitrarily small force . In a complex inner-product space, apply the same argument to to obtain the imaginary part too. Conversely, if is orthogonal to the subspace, the Pythagorean theorem for inner product spaces gives
Thus orthogonality characterizes the unique best approximation:
Finite dimension also guarantees existence by solving the invertible Gram matrix normal equations; completeness of the ambient inner-product space is unnecessary.
Use the convention that the inner product is conjugate-linear in its first argument; the real case is the same without conjugates. Linear independence makes the Gram matrix positive definite, so exists and is Hermitian. Define the Riesz dual basis in an inner product space within by
Then . Its own Gram entries are
Hence the inverse Gram matrix is the Gram matrix of the dual vectors:
The dual vectors are required to lie in ; adding arbitrary vectors orthogonal to that subspace would preserve the displayed biorthogonality but would destroy this conclusion.
An abstract root system, in the reduced crystallographic convention, is a finite subset of a finite-dimensional real Euclidean space satisfying the following conditions:
  • and spans .
  • For each , ; this is the reduced condition.
  • The root reflection preserves .
  • Every Cartan integer is an integer.
The root system is irreducible if it is not a union of two nonempty mutually orthogonal root systems. The integrality condition is essential for the finite angle list below; arbitrary finite reflection systems need not satisfy it.
The possible angles are , , , , , , , , and . For roots , put
Both are integers, they have the same sign when nonzero, and
If the roots are linearly independent, strict Cauchy-Schwarz inequality gives . Hence . These values give, respectively,
For the last column, the possible absolute pairs are , , and , up to interchange, and . Parallel roots are equal or opposite by reducedness, giving or . Thus the list is exhaustive, and the rank-two root systems realize each nonparallel possibility.
Now suppose two simple roots have angle . Distinct simple roots have nonpositive inner product. Here is a proof from their defining sign property: if , then with a positive Cartan integer . This would be a root whose coefficients in the root basis include both a positive and a negative coefficient, which is impossible. Therefore, after normalizing the simple roots to unit length, any nonzero off-diagonal inner product is at most , by the angle list.
Let , , so . If there were a third simple root with unit vector , write and . Independence of the simple roots makes their Gram matrix positive definite, so
If either or were nonzero, its absolute value would be at least . Since , the displayed determinant would then be nonpositive. Thus . Every other simple root is orthogonal to both and : a triple bond isolates a G2 component.
Indeed, the root reflections in preserve their plane and fix its orthogonal complement; the root reflections in the other simple roots preserve that complement and fix the plane. The given facts that these reflections generate the Weyl group and that every root is Weyl group-conjugate to a simple root imply that every root lies entirely in one of those two orthogonal subspaces. If there were any other simple root, this would make reducible. Therefore irreducibility forces rank two.
Up to interchanging the simple roots, take short and long. The squared-length ratio is . Choose a scale and orthonormal coordinates so that
The Cartan integers are and . In coefficient coordinates , the two root reflections are
Their orbits of and are, respectively,
These sets are closed under both displayed reflections; for example , , , and . The assumption that every root is conjugate to a simple root now determines all twelve roots, with no further ones possible.
Figure 1.
The twelve G2 roots, with the six positive roots labelled and the simple roots emphasized
.
For completeness, this twelve-element set really is a root system. It is finite, spans the plane, and is reduced. The six short roots have length and directions spaced by ; the six long roots have length and interleaved directions, also spaced by . Their mutual Cartan integers are integral: within either length class they are , , or as applicable, and between the two length classes they are , , or . Closure under all root reflections follows by conjugating the two simple reflections, since every listed root lies in a simple-root orbit. The nonorthogonal simple roots make it irreducible. We have therefore proved existence as well as uniqueness, up to isometry and overall scale:
A flat torus is the quotient , where is a full-rank Euclidean lattice, with the Riemannian metric induced from the Euclidean metric. Translation by a lattice vector is a Riemannian isometry, so this metric is well defined and has zero curvature. Its volume is the covolume of .
The dual lattice is . For , the function descends to the flat torus, and direct differentiation gives
Conversely, write and . Ordinary Fourier series on have frequencies , corresponding to . After normalization they form a complete orthonormal basis in , and the Fourier coefficients of a smooth function decay rapidly. Applying the Laplace-Beltrami operator term by term shows that a smooth eigenfunction with eigenvalue has nonzero coefficients only where . Thus the spectrum of a flat torus is
The formula counts multiplicity over either the real or complex numbers: the two frequencies give the real sine and cosine functions.
For the rigidity assertion in dimension two, it suffices to reconstruct a rank-two Euclidean lattice from its vector-length multiset. The multiset first determines its covolume . Indeed, a bounded fundamental parallelogram, and comparison of the cells meeting a disk with slightly larger and smaller disks, give the lattice-point asymptotics by fundamental cells
The spectrum determines , including multiplicities, so it determines .
Let be the smallest nonzero vector length, and choose with . This vector is primitive: with would contradict minimality. From the length multiset subtract the two vectors at every positive length , for . The shortest remaining length is exactly the length of a shortest vector . This subtraction is legitimate even if several directions have length : it subtracts just two copies, and then . It also proves that is independent of our choice of shortest direction. Replace by to arrange
The replacement stays outside and cannot be shorter than , so a reduced choice still has length .
The shortest-independent-vector basis lemma says that are a basis of . To prove it, choose coordinates with . Because is primitive, ; let be the smallest positive vertical coordinate in . Write , with after changing its sign. If , a vector of vertical coordinate , reduced horizontally modulo , has
This contradicts the definition of , since . Hence and generate .
The Gram matrix of this basis is , with , and its determinant is . Therefore
Changing the sign of one basis vector allows . The numbers , all audible from the spectrum, consequently determine the Gram matrix, and hence , up to an orthogonal transformation. Taking dual lattices gives the same conclusion for . Isospectral flat two-dimensional tori are isometric. The argument uses the special basis property of rank-two Euclidean lattices, so it does not claim higher-dimensional rigidity.
Put . The Debiased Lasso correction is
For the Nodewise Lasso, let . Its Karush-Kuhn-Tucker conditions are
so at a nonzero coefficient and at a zero coefficient. Multiplying by gives the nodewise-Lasso residual identity
We require nonzero design columns, a necessary qualification missing from the unrestricted printed formulation. Because , would force both and , and hence . Thus nonzero columns suffice for even when or the full Gram matrix is singular. A zero column instead gives and its coefficient is unidentifiable from ; the proposed normalization and confidence interval do not exist there.
For each , define by and . Set row of to . Since ,
This row construction is the nodewise normalization used by van de Geer and coauthors. The matrix need not be symmetric or be an actual inverse.
Substitute the linear model into the correction. The exact decomposition is
In the fixed-design normal linear model, or with and nodewise estimates and tuning chosen from alone, a linear transformation of a multivariate normal distribution gives . Marginal normality of without this conditional assumption would not suffice for a response-dependent random design. No independence between and is asserted. The diagonal cancellation and the Holder inequality give the debiased-Lasso remainder bound
For a coordinate variance, , whence
Writing for the corresponding standard normal quantile, an approximate confidence interval is
Equivalently its half-width is . We assume and . For asymptotic coverage, the standardized remainder must have convergence in probability to zero and the coordinate variance must be nondegenerate. Exact conditional normality of alone does not give finite-sample exact coverage for .
One sufficient random-design regime is the Gaussian identity design for debiased Lasso: take independent rows , independently of , fixed , , , and
Choose and all for sufficiently large fixed . Here is why this regime has the requested remainder bound. Uniformly over columns, their squared norms divided by tend to one, while all off-diagonal sample cross-correlations have magnitude at most a constant times with probability tending to one. Taking larger makes the zero vector satisfy all nodewise Karush-Kuhn-Tucker conditions strictly, so for every on that event, and eventually. This also gives uniformly.
In this regime, the entrywise entrywise concentration of a Gaussian Gram matrix and Lasso cone condition argument in Gaussian identity design for debiased Lasso gives a Compatibility condition for the Lasso bounded away from zero with probability tending to one. Together with the noise-score event and the Basic inequality for the Lasso, this yields, for a fixed ,
For example, the cone inequality and compatibility bound give on their joint event. Therefore, with a fixed ,
The stated sparsity condition makes this remainder vanish and supplies the preceding confidence-interval requirement. Both dimensions may grow: and satisfy these assumptions. The concentration and Gaussian-design compatibility results are sufficient standard conditions here; the question does not require their proofs. The displayed rate is a high-probability fixed-constant bound, rather than only a claim of boundedness in probability.