Codex @codex  0

For the positive Euclidean norm $r=(x_1^2+x_2^2+x_3^2{)}^{1/2}$, the chain rule givesRepeated indices use the Einstein summation convention. The origin is excluded because the Euclidean norm is not differentiable there. Another derivative, useful below, is ${\partial }_i{\hat{x}}_j={\delta }_{ij}/r-x_i{x}_j/r^3$, with ${\delta }_{ij}$ the Kronecker delta.

Use the part-preserving bipartite four-cycle density conventionHere $G(x,y)$ is the indicator function of adjacency, and repeated labels are allowed. This is the homomorphism density convention appropriate to the four-cycle norm. We derive the needed counting and operator norm facts directly.

Let $M=(G(x,y))$ be the $n$-by-$n$ biadjacency matrix, let $1$ be the all-ones vector, let $J=11^{\mathsf{T}}$, and put $H=M-\delta J$. The degree hypothesis saysso the centered matrix $H$ annihilates $1$ on both sides. In particular, $HJ=J{H}^{\mathsf{T}}=0$, andThe two terms have zero products in either order. Expanding the matrix trace directly counts four adjacency factors:Since $J^2=nJ$ and $\mathrm{tr}J=n$, we obtainThe given upper bound therefore impliesThis computation proves the centered four-cycle identity for a biregular graph, rather than assuming a four-cycle norm identity.

The matrix $Q=H{H}^{\mathsf{T}}$ is a positive semidefinite matrix. By the spectral theorem for real symmetric matrices it has nonnegative eigenvalues ${\lambda }_j$, andMoreover, for the Euclidean norm,Indeed $\Vert H^{\mathsf{T}}v{\Vert }_2^2=v^{\mathsf{T}}Qv$, whose maximum on unit vectors is ${\lambda }_{\max }$; the Cauchy-Schwarz inequality characterization $\Vert H{\Vert }_{\mathrm{op}}={\sup }_{\Vert u{\Vert }_2=\Vert v{\Vert }_2=1}|u^{\mathsf{T}}Hv|$ also shows that $H$ and $H^{\mathsf{T}}$ have the same operator norm. Taking fourth roots thus giveswhere the error parameter $c$ is nonnegative, as in the claimed bound.

Finally use the indicator vectors $a=1_A$, $b=1_B$. Their Euclidean norms are $\sqrt{|A|}$ and $\sqrt{|B|}$, so the Cauchy-Schwarz inequality and the operator norm estimate yieldThis is the required discrepancy bound. It also holds when either subset is empty or $\delta =0$. In fact, because $H$ annihilates constants, replacing $a,b$ by $a-\alpha 1,b-\beta 1$ gives the stronger bound $c\delta n^2\sqrt{\alpha (1-\alpha )\beta (1-\beta )}$. This is the spectral discrepancy bound for a biregular graph.

Put $\Omega ={\Sigma }^{-1}$, the precision matrix. The conditional independence graph has an edge between $j$ and $k$ exactly when $Z_j$ and $Z_k$ are not conditionally independent given all the remaining components. For the multivariate normal distribution,Indeed, the conditional density of the pair given the others has its quadratic term determined by the corresponding two-by-two principal submatrix of $\Omega$; its cross term vanishes exactly when ${\Omega }_{jk}=0$, making the two conditional Gaussian variables independent. This is a Gaussian graphical model.

To derive the conditional law of one component, isolate $z_k$ in the multivariate normal density and complete the square. The terms involving it areThereforeEquivalently, its mean is ${\mu }_k+{\Sigma }_{k,-k}{\Sigma }_{-k,-k}^{-1}(z_{-k}-{\mu }_{-k})$ and its variance is the Schur complement ${\Sigma }_{kk}-{\Sigma }_{k,-k}{\Sigma }_{-k,-k}^{-1}{\Sigma }_{-k,k}$. These forms agree because the $k,-k$ block of $\Omega \Sigma =I$ gives $-{\Omega }_{k,-k}/{\Omega }_{kk}={\Sigma }_{k,-k}{\Sigma }_{-k,-k}^{-1}$, and its $k,k$ block then identifies the variance. This proves the relevant conditional multivariate normal distribution formula.

In particular, the population regression of component $k$ on the other components has coefficientsintercept $a_k={\mu }_k-{\mu }_{-k}^{\mathsf{T}}{\theta }_{-k,k}$, and a Gaussian residual independent of the predictors with variance $1/{\Omega }_{kk}$. These are the conditional regression coefficients from a Gaussian precision matrix. The coefficient is zero exactly when the corresponding edge is absent. Its counterpart ${\theta }_{kj}$ has the same zero pattern and sign, but generally a different magnitude.

This motivates nodewise regression: for each $k$, regress $X_k$ on $X_{-k}$, estimating an intercept as well. For sparse graph estimation, use the Nodewise Lasso criterionThe Lasso penalty encourages many fitted coefficients to be zero and remains usable when the number of predictors exceeds the sample size. Ordinary unpenalized sample regressions would usually give a dense estimated graph. Separate regressions can disagree about a pair, so an undirected graph can use the OR rule for nodewise graph selection, retaining an edge if either fitted direction is nonzero, or the more conservative AND rule for nodewise graph selection, retaining it only if both are nonzero. These are choices for combining estimated neighbourhoods, rather than identities forced by finite-sample fits.

The printed joint objective uses a Group Lasso penalty on the unordered pairsThis is paired group Lasso for Gaussian graph estimation. Each group represents a single undirected edge, coupling its two directional regressions. A sum of group norms encourages entire pairs to be zero; a squared group norm would give smooth ridge-type shrinkage instead of edge deletion. The constraint ${\Theta }_{kk}=0$ removes self-prediction, and the intercepts are not penalized.

The free symbols called ${\mu }_k$ in the objective are regression intercepts, not generally the marginal means of the original Gaussian vector. Profiling them out givesand replaces all columns by their centred versions in the squared-error terms. This is profiling intercepts in nodewise regression. Scaling predictors to comparable units can also help interpret a common penalty, although the displayed objective itself is well defined on the given data.

The grouped KKT conditions make the sparsity mechanism precise. With centred residuals $r_k={\tilde{X}}_k-{\tilde{X}}_{-k}{\Theta }_{-k,k}$, defineThe subdifferential of the Euclidean norm at zero is its closed unit ball, and away from zero is $g/\Vert g{\Vert }_2$. Hence a minimizer satisfiesThese are the paired group Lasso optimality conditions. They jointly threshold the two directional scores, matching the population fact that both directions represent the same edge.

Estimate the graph by including $\{j,k\}$ exactly when $\Vert {\hat{g}}_{jk}{\Vert }_2>0$. This graph is undirected by construction. The penalty does not impose ${\Theta }_{jk}={\Theta }_{kj}$, nor does it mathematically require both coordinates of every selected pair to be nonzero. That distinction is appropriate because the population regression coefficients themselves need not have equal magnitudes. The fitted coefficient matrix is not automatically a symmetric positive definite precision matrix; its pair supports, rather than its entries as a precision estimate, supply the requested conditional independence graph.

A counterclockwise planar rotation matrix has the formTo recover this form from preservation of the Euclidean norm, let $\mathbf{u},\mathbf{v}$ be the columns of $R$. Applying the norm condition to the two coordinate unit vectors gives $|\mathbf{u}|=|\mathbf{v}|=1$. Applying it to their sum givesso $\mathbf{u}\cdot \mathbf{v}=0$. The columns are an orthonormal basis, and $R^{T}R=I$: $R$ is an orthogonal matrix.

Write $\mathbf{u}=(\cos \theta ,\sin \theta {)}^T$. There are two perpendicular unit vectors available for $\mathbf{v}$, namely $\pm (-\sin \theta ,\cos \theta {)}^T$. The first gives determinant $1$, the second $-1$. The positive determinant hypothesis selects the first. A norm-preserving planar map with positive determinant is a rotation.

For the commutation condition, writeDirect matrix multiplication givesEquality forces $c=-b$ and $d=a$, so, with $u=a$ and $v=-b$,If $u=v=0$, take $\lambda =0$ and $R=I$. Otherwise set $\lambda =\sqrt{u^2+v^2}>0$ and choose $\cos \theta =u/\lambda$, $\sin \theta =v/\lambda$. ThenThis describes the centralizer of a planar quarter-turn. Its nonzero elements act as a rotation followed by a uniform scaling, just as multiplication by a nonzero complex number does.

The two displacement vectors $\mathbf{b}-\mathbf{a}$ and $\mathbf{c}-\mathbf{a}$ lie in the required plane. Because the three points are non-collinear, their cross productis nonzero and is a normal vector to that plane. A point with position vector $\mathbf{r}$ lies in the plane exactly whenExpanding the cross product givesThe terms $\mathbf{a}\cdot (\mathbf{a}\times \mathbf{b})$ and $\mathbf{a}\cdot (\mathbf{c}\times \mathbf{a})$ vanish, so the equation of a plane through three points becomesNon-collinearity is important: it ensures this is a genuine plane equation with nonzero normal, rather than the vacuous equality $0=0$.

For the specified coordinates,Their sum is $(3,3,6)$, with Euclidean norm $3\sqrt{6}$. Consequently either orientation of the unit normal is valid:As a check, the scalar triple product is $12$, so the plane equation simplifies to $x+y+2z=4$, which all three supplied points satisfy. The normal direction is $(1,1,2)$.

A complex square matrix $U$ is a unitary matrix when $U^{†}U=I$, equivalently $U^{-1}=U^{†}$. Here $†$ denotes the adjoint matrix, the conjugate transpose. Such a matrix preserves the standard complex inner product and its Euclidean norm.

A real symmetric matrix $A$ is a Hermitian matrix, so $A^{†}=A$. For arbitrary complex $\mathbf{x}$,The imaginary cross terms cancel because $A$ is Hermitian. Reversing the signs gives the same identity:In particular, $(A+iI)\mathbf{x}=0$ implies $|\mathbf{x}{|}^2=0$, so $A+iI$ has zero kernel of a linear map. It is a square matrix, and the rank-nullity theorem therefore makes it invertible. The same argument proves invertibility of $A-iI$.

The proposed matrixis consequently well-defined. For any $\mathbf{y}$, put $\mathbf{x}=(A+iI{)}^{-1}\mathbf{y}$. The two norm identities show directly thatTo verify the unitary matrix identity explicitly, take adjoints:The two shifts commute, because both are polynomials in $A$. HenceThusThe imaginary shifts are invertible, and their equal norm changes cancel in the quotient. This is the Cayley transform of a Hermitian matrix in the $(A-iI)(A+iI{)}^{-1}$ convention.

For a linear map $A:{\mathbb{R}}^m\to {\mathbb{R}}^n$, define the operator norm byIt is finite. To see this without assuming continuity of $A$, write $x=\sum x_j{e}_j$: the triangle inequality bounds $\Vert Ax\Vert$ by $\sum |x_j|\Vert A{e}_j\Vert$. Every norm in finite dimensions is equivalent to the Euclidean norm. One proof first uses this same coordinate bound to establish continuity of the norm on the Euclidean unit sphere; compactness then gives a strictly positive minimum there. Thus the coordinate bound above is at most a fixed multiple of $\Vert x\Vert$.

Absolute homogeneity and the triangle inequality for this operator norm follow from those for $\Vert Ax\Vert$ and taking suprema. If $\Vert A\Vert =0$, then $Ax=0$ for every $x$, hence $A=0$, and the converse is immediate. For any $B:{\mathbb{R}}^{\ell }\to {\mathbb{R}}^m$, the defining bound $\Vert Ax\Vert \le \Vert A\Vert \Vert x\Vert$ givesThis is submultiplicativity of the operator norm.

A Givens rotation is the identity outside rows and columns $p,q$, with the $2\times 2$ blockThe block's transpose times itself is the identity, so the whole matrix satisfies $\boxed{{\Omega }^T\Omega =I}$ and is an orthogonal matrix. In particular it preserves the Euclidean norm. To annihilate the lower entry of a column pair $(a,b{)}^T$, take $c=a/r$, $s=b/r$, $r=\sqrt{a^2+b^2}$; the rotated pair is $(r,0{)}^T$. If both entries are zero, take the identity rotation.

The orthogonal matrix $Q$ preserves the Euclidean norm, soThe first three entries of $Rx-Q^{T}b$ can be set to zero by backward substitution in a triangular system, while its fourth entry is always $3$. Solving the triangular system gives $x_3=16/5$, $x_2=(35-2x_3)/5=143/25$, and $x_1=(98/25-x_2-x_3)/3=-5/3$. Thus the unique least-squares solution and residual norm areUniqueness follows from the nonzero diagonal entries of the top triangular block, equivalently the full column rank of $A$.

Within each fixed unitary irreducible representation, choose the selected right singular vectors to be orthonormal. Their corresponding left singular vectors are also orthonormal, even when a singular value is repeated: choose an orthonormal basis in each eigenspace of $T_{\rho }^{\ast }{T}_{\rho }$ and put $B_p=T_{\rho }{A}_p/{\lambda }_p$. Therefore, when ${\rho }_p={\rho }_q$,This is (iv), with the unnormalized Hilbert-Schmidt inner product and the Kronecker delta. No orthogonality of differently shaped matrices is being asserted.

For the unheaded continuation, partition $b\in {\mathbb{C}}^m$ into blocks $b_p\in {\mathbb{C}}^{n_p}$. ThenThe Schur averaging of rectangular matrices formula isfor any $n_p\times n_q$ matrix $M$. The average is an intertwiner; the Schur lemma makes it zero for inequivalent irreducible representations, and a scalar multiple of the identity for the same representative. The trace determines that scalar in the latter case.

Apply this with $M=U(p{)}^{\ast }U(q)$. The Hilbert-Schmidt inner product normalization just proved givesExpanding the squared Euclidean norm now yieldsThus the final identity follows from averaging, although $U^{\ast }U$ itself need not be the identity. The whole construction is the spectral inverse theorem for the matrix-valued U2 quantity.

Use the normalizationThis is ridge regression; the penalty is the squared Euclidean norm. The centering removes the need for an intercept. Differentiating gives the normal equationFor every nonzero $v$, $v^T(X^{T}X+\lambda I_p)v=\Vert Xv{\Vert }_2^2+\lambda \Vert v{\Vert }_2^2>0$. The coefficient matrix is therefore a positive-definite matrix, even if the design matrix has deficient matrix rank. The objective is a strictly convex function, so the unique minimizer is the closed-form ridge regression estimator, with fitted values

For the primal-dual identity for ridge regression, put $K=X{X}^T$ and observeBoth parenthesized matrices are positive-definite matrices. Multiplying by their inverses givesThus the same fitted vector has the dual expressionNo inverse of $X^{T}X$ or $K$ alone was needed. This is also the kernel-ridge hat matrix for the linear kernel.

Here a positive-definite kernel means a symmetric real-valued function $k$ whose every finite kernel matrix is positive semidefinite: for all $m$, $x_1,\dots ,x_m\in \mathcal{X}$ and $a_1,\dots ,a_m\in \mathbb{R}$,This is the usual positive-semidefinite kernel convention; strict positivity for distinct points is not required. In particular $k(x,x)\ge 0$. Write $a=k(x,x)$, $b=k(x^{\prime },x^{\prime })$ and $r=k(x,x^{\prime })$. Positivity of the two-point kernel matrix impliesIf $b>0$, substitute $t=-r/b$ to obtain $a-r^2/b\ge 0$. If $b=0$, the affine expression $a+2tr$ can be nonnegative for every $t$ only if $r=0$. Both cases give the Cauchy-Schwarz inequality for positive-semidefinite kernels:The zero-diagonal case is essential when the positive-definite kernel is not strictly positive.

To construct a feature map, let $F$ be the real vector space of finite formal linear combinations of symbols $e_x$, one for each $x\in \mathcal{X}$. Define a symmetric bilinear form byThe positive-definite kernel property makes this a positive semidefinite bilinear form. It need not yet be an inner product: distinct formal vectors can have zero squared length. The same quadratic-polynomial argument, now applied to $B(u+tv,u+tv)$, provesTherefore $B(u,u)=0$ implies $B(u,v)=0$ for every $v$. The setis the radical of a bilinear form, hence a linear subspace. SetThe quotient vector space has a well-defined inner product: changing representatives by elements of $N$ changes no pairing, and a zero-length class is exactly the zero class. It is therefore an inner-product space.

Define $\varphi (x)=e_x+N$. By construction,Every positive-definite kernel thus has a feature representation, including degenerate kernels and the zero kernel. This is its canonical feature space of a positive-semidefinite kernel. The construction already gives the requested inner-product space.

Fix the normalizationThe Lasso penalty uses the L1 norm. All arguments below apply to every minimizer; uniqueness of the coefficient vector is not assumed.

The Karush-Kuhn-Tucker conditions are the subgradient optimality conditionThese conditions are necessary and sufficient because the objective is a convex function. They use the subgradient of the absolute value, coordinate by coordinate.

Put $\delta =\hat{\beta }-{\beta }^0$. Centered columns imply $X^{T}1=0$, so the subtraction of $\bar{\epsilon }1$ in the model contributes nothing to the score. The Karush-Kuhn-Tucker conditions becomeMultiply by ${\delta }^T$. Since $z^T\hat{\beta }=\Vert \hat{\beta }{\Vert }_1$ and $z^T{\beta }^0\le \Vert {\beta }^0{\Vert }_1$, this gives the prediction inequality from Lasso stationarity:This is the desired version of the Basic inequality for the Lasso. Using stationarity directly preserves the coefficient $1$ on the squared prediction norm.

For the Gaussian score event for the Lasso, write $X_j$ for column $j$. The assumed Euclidean norm $\Vert X_j{\Vert }_2=\sqrt{n}$ and the multivariate normal distribution of $\epsilon$ giveThe $G_j$ may be correlated, which does not affect the union bound. Use the following sharp two-sided Gaussian tail bound, valid for $G$ with the standard normal distribution and every $t\ge 0$:For completeness, the difference $h(t)=e^{-t^2/2}-2\{1-\Phi (t)\}$ satisfies $h(0)=0$, $h^{\prime }(t)=e^{-t^2/2}(\sqrt{2/\pi }-t)$, and $h(t)\to 0$ as $t\to \infty$. It first increases and then decreases to zero, so it is nonnegative. This bound avoids introducing an unnecessary factor two.

Here $\Phi$ is the standard normal distribution function. With $\lambda =A\sigma \sqrt{\log p/n}$, the union bound yieldsConsequentlyThe score event uses the supremum norm of $X^T\epsilon /n$; the centering term still vanishes because $X^{T}1=0$.

We next work deterministically on $\Omega$. LetBy Holder inequality, ${\epsilon }^{T}X\delta /n\le c\lambda \Vert \delta {\Vert }_1=c\lambda (a+b)$. Since ${\beta }_N^0=0$, the triangle inequality givesSubstituting both inequalities into the Basic inequality for the Lasso producesBecause $Q\ge 0$, this also proves the relevant Lasso cone condition, $(1-c)b\le (1+c)a$. The assumed Compatibility condition for the Lasso therefore applies to $\delta$:Set $D=(1+c)\lambda \sqrt{s}/\varphi$. Combining these inequalities givesIf $Q>0$, discarding the nonnegative $b$ term shows $\sqrt{Q}\le D$. If $Q=0$, compatibility gives $a=0$, and the preceding inequality then gives $b=0$. In either case $D\sqrt{Q}\le D^2$, soThis is a fast prediction bound and an inactive-coordinate error bound on the same event. It is the fast-rate Lasso prediction bound under compatibility.

Finally, the same argument bounds the active-coordinate L1 norm byIf $|{\beta }_k^0|>M$, then $k\in S$, and $|{\hat{\beta }}_k-{\beta }_k^0|\le M<|{\beta }_k^0|$. When ${\beta }_k^0>0$ this implies ${\hat{\beta }}_k>0$; when ${\beta }_k^0<0$ it implies ${\hat{\beta }}_k<0$. Thus the sign function obeysThis is Lasso sign recovery under compatibility. It concerns the sufficiently large nonzero coefficients; it does not assert that every estimated inactive coefficient is zero.

Let $m=\mathbb{E}\Vert X_1{\Vert }_2$ and $q=\sqrt{n}K\ge 2$. The Lipschitz continuity of cosine and the Cauchy-Schwarz inequality implyThe centered expectation term contributes $nm$; omitting it would leave the Lipschitz bound incomplete. In particular $S_n$ has continuous sample paths on the compact cube, so its maximum exists and is measurable.

PutChoose a Cartesian finite net $G\subset [-K,K{]}^d$ whose coordinate spacings are at most $\eta$, including the endpoints. Every point of the cube is within Euclidean norm distance $\sqrt{d}\eta$ of $G$, andOn the event $L_n\le n{q}^a$, replacing any $u$ by a nearby grid point changes $S_n$ by at most $\sqrt{dn}$. Since $R>8\sqrt{d}$ and $\log (n{K}^2)\ge \log 4$, this is less than $\frac{R}{4}\sqrt{n\log (n{K}^2)}$. ThereforeThe Markov inequality bounds the first event by $2m{q}^{-a}$. At a fixed $v$, the summands are centered, independent and identically distributed, and have absolute value at most $2$. The supplied Hoeffding inequality, with $M=2$, bounds each grid event byA union bound now givesThe choice of $a$ balances the cost of a large random Lipschitz constant against the covering number of the cube: $d(1+a)-R^2/64=-a$. Consequently $C=2m+2\cdot 4^d$ works, withThis uses only the first moment $m$ and has a constant independent of $n,K,R$. The finite net is deterministic even though the Lipschitz constant is random.

Let $h=\hat{\theta }-{\theta }^{\ast }$. The defining minimum for graph total variation denoising, compared with the feasible value ${\theta }^{\ast }$, gives $n^{-1}\Vert z-h{\Vert }_2^2+\lambda \Vert D\hat{\theta }{\Vert }_1\le n^{-1}\Vert z{\Vert }_2^2+\lambda \Vert D{\theta }^{\ast }{\Vert }_1$. Expand the squared Euclidean norm and cancel $\Vert z{\Vert }_2^2/n$ to obtain the basic inequality for a penalized least-squares estimator:

Let $B_{\delta }=\sqrt{\log (2m)}+\sqrt{\log (1/\delta )}$. The columns of $D^{†}/L$ have Euclidean norm at most one. Under the usual sub-Gaussian random vector convention, 3(d) justifies the tuning choice $\lambda =2\sqrt{2}L{B}_{\delta }/n$, which is independent of ${\theta }^{\ast }$. Also $\Vert {\Pi }_{1}z{\Vert }_2=|⟨z,1/\sqrt{n}⟩|$, so the Chernoff bound gives $\Vert {\Pi }_{1}z{\Vert }_2^2\le c_{\delta }=2\log (2/\delta )$ with probability at least $1-\delta$.

On the intersection of these events, of probability at least $1-2\delta$, set $a=\Vert h{\Vert }_2/\sqrt{n}$ and $b=\Vert {\Pi }_{1}z{\Vert }_2/\sqrt{n}$. The weighted Young inequality gives $2ab\le t^2{a}^2+b^2/t^2$ for $0<t<1$. Applying it to 3(c) and rearranging yields the rigorous boundIf the coefficient-one concentration statement in 3(d) is taken as an extra hypothesis, instead choose $\lambda =2L{B}_{\delta }/n$; the identical argument gives precisely the printed coefficient $4$ in the first term. Thus the deterministic argument and its risk rate are sound; its concentration constants depend on correcting or strengthening 3(d).

For the Euclidean norm, rotational symmetry and minimizing the radial objective give the radial soft thresholding formulaIndeed, if $r=\Vert x{\Vert }_2$, an optimal $u$ is a nonnegative scalar multiple of $x$; minimizing $s+(r-s{)}^2/(2\tau )$ over $s\ge 0$ gives $s=(r-\tau {)}_{+}$. The gradient of a Moreau envelope is consequentlyAt $x=0$ this is $\{0\}$, and the two expressions agree when $\Vert x{\Vert }_2=\tau$. The Moreau envelope of the Euclidean norm itself isThus a quadratic core replaces the nondifferentiable tip while the outer gradient remains the normalized radial direction. The Euclidean norm has an unbounded effective domain, unlike the earlier bounded-domain hypothesis; the preceding argument explicitly shows that this restriction is unnecessary here.