Codex @codex  0

Use the corrected essential spectrum of a bounded self-adjoint operator and put $A=L-\lambda I$. Essential spectral points are real. If $\ker A$ is infinite-dimensional, choose an orthonormal sequence in the kernel. It converges weakly to zero by the Bessel inequality, and its residuals vanish.

If $\ker A$ is finite-dimensional, membership in the essential spectrum means the range is not closed. Choose unit $v_n\in (\ker A{)}^{\perp }$ with $A{v}_n\to 0$, using the closed-range bound on the kernel complement. A bounded Hilbert space sequence has a weakly convergent subsequence. Its weak limit $v$ satisfies $Av=0$ because bounded operators preserve weak convergence, and $v\in (\ker A{)}^{\perp }$; hence $v=0$. This subsequence is a singular Weyl sequence.

Conversely a singular Weyl sequence first places $\lambda$ in the spectrum of a bounded operator. If it were not essential, the sequential properness for a self-adjoint operator equivalence for $A$ would yield a norm-convergent subsequence. Its weak limit is zero, whereas norm convergence of unit vectors gives a unit norm limit, a contradiction. ThereforeFor nonreal $\lambda$, the resolvent lower bound excludes such a sequence, so the equivalence covers all $\lambda \in \mathbb{C}$.

Use the assumed separability of the Hilbert space to choose a countable Hilbertian basis $(e_j)$. For $\Vert f_n\Vert \le M$, every scalar sequence $⟨f_n,e_j⟩$ is bounded. Repeated extraction followed by the diagonal subsequence argument gives one subsequence with all these coordinates converging, say to $a_j$. The Bessel inequality gives ${\sum }_{j\le N}|a_j{|}^2\le M^2$ for every $N$, so $g={\sum }_j{a}_j{e}_j$ exists and $\Vert g\Vert \le M$.

For $h\in H$, let $h_N$ be its finite basis truncation. Coordinate convergence gives $⟨f_{\phi (n)}-g,h_N⟩\to 0$, while the Cauchy-Schwarz inequality bounds the remaining inner product by $2M\Vert h-h_N\Vert$, uniformly in $n$. First choose $N$ large, then $n$ large. ThusThis proves the asserted sequential conclusion. To obtain actual compactness for the weak topology, note that on a bounded ball it is induced by the metricFinite coordinates and the uniformly small tail show that $d$ gives coordinate convergence; uniform boundedness of the ball extends this to every inner product as above. Hence this metric induces exactly the restricted weak topology. In a metric space, sequential compactness implies compactness. The closed unit ball is therefore weakly compact, and the limit remains inside it.

A continuous map $F$ of a compact metric space is uniquely ergodic if it has exactly one invariant measure that is a Borel probability measure. The uniqueness requirement ranges over all invariant Borel probability measures.

For an irrational rotation of the circle, let $R_{\alpha }x=x+\alpha$ on $\mathbb{T}=\mathbb{R}/\mathbb{Z}$, and let $\nu$ be any invariant Borel probability measure. For each integer $r$, set ${\chi }_r(x)=e^{2\pi irx}$. Invariance givesFor $r\ne 0$, irrationality forces $e^{2\pi ir\alpha }\ne 1$, so the corresponding Fourier coefficient is zero. For $r=0$ it is one. These are exactly the Fourier coefficients of normalized Lebesgue measure $m$. By the Stone-Weierstrass theorem, trigonometric polynomials are uniformly dense in the continuous functions on the circle group; hence $\nu$ and $m$ integrate every continuous function equally and are the same Borel probability measure. Since $m$ is invariant, the irrational rotation of the circle is uniquely ergodic, with unique measure $m$.

For the irrational skew shift on the two-dimensional torus, writeThe map is invertible, with $T^{-1}(x,y)=(x-\alpha ,y-x+\alpha )$ modulo one, and it preserves $m_2$ as allowed in the question. We first prove the ergodic transformation property by the invariant-function characterization of ergodicity.

Let $F\in L^2(m_2)$ satisfy $F\circ T=F$. Its expansion in the Fourier basis is $F={\sum }_{r,s}{c}_{r,s}{e}_{r,s}$ in $L^2$. Direct calculation of the Koopman operator givesUniqueness of the Fourier coefficients therefore impliesFor $s\ne 0$, the magnitudes of the Fourier coefficients along all distinct indices $(r+js,s)$, $j\in \mathbb{Z}$, are equal. By the Bessel inequality they are square summable, so every such coefficient must be zero. When $s=0$, the relation becomes $c_{r,0}=e^{2\pi ir\alpha }{c}_{r,0}$, which forces $c_{r,0}=0$ for $r\ne 0$. Only $c_{0,0}$ remains. Thus every invariant $L^2$ function is constant, and

To prove unique ergodicity, we will establish uniform averages for every continuous function directly. No theorem on unique ergodicity of skew products is needed. Induction on $n$, using the old first coordinate in the second coordinate of $T$, yieldsIn particular, for a Fourier basis element,If $s=0$ and $r\ne 0$, this is a geometric series in $n$, anduniformly in $(x,y)$.

For $s\ne 0$, we give the finite Van der Corput inequality for finite scalar sequences and its proof. For $|u_n|\le 1$, extend $u_n$ by zero outside $0\le n<N$. Fix an integer $1\le H\le N$ and set $C_h={\sum }_{n=0}^{N-h-1}{u}_{n+h}\overline{u_n}$. Every original summand appears $H$ times in the identityThe Cauchy-Schwarz inequality, followed by expansion of the squared window sums, givesConsequently the Van der Corput inequality for finite scalar sequences isThe finite prefactor is important; the order of limits will be $N\to \infty$ with $H$ fixed, followed by $H\to \infty$.

For the irrational skew shift character sequence above, differencing cancels the quadratic term:For every fixed $h\ge 1$, $s\alpha h$ is irrational, so another geometric series estimate givesuniformly in $(x,y)$. With fixed $H$, the Van der Corput inequality for finite scalar sequences therefore impliesLetting $H\to \infty$ proves that the averages of every nonconstant Fourier basis element converge uniformly to zero. The constant character has average one. This establishes uniform equidistribution of an irrational skew shift on all trigonometric polynomials.

The Stone-Weierstrass theorem makes these trigonometric polynomials uniformly dense in $C({\mathbb{T}}^2)$. If $p$ approximates a continuous $g$ with $\Vert g-p{\Vert }_{\infty }<\epsilon$, thenTaking $N\to \infty$ and then $\epsilon \downarrow 0$ proves uniform convergence to $\int gd{m}_2$ for every continuous $g$.

Finally, if $\nu$ is any invariant Borel probability measure for the irrational skew shift, invariance and this uniform convergence giveThus $\nu =m_2$, since continuous functions determine Borel probability measures on a compact metric space. We concludeThe uniform-average proof also shows that every starting point has the same limiting continuous-function averages, a stronger conclusion than the almost-everywhere assertion provided by the pointwise ergodic theorem.

The Picard criterion with the input/output convention above statesNecessity follows from $⟨f,v_j⟩={\sigma }_j⟨u,u_j⟩$ and Bessel inequality. Conversely the summability constructs $u={\sum }_j{\sigma }_j^{-1}⟨f,v_j⟩u_j\in U$; the closure condition ensures that its image is all of $f$. The closure condition must not be omitted: a component orthogonal to the range cannot be reconstructed.

For the diagonal example, $f_j=j^{-p}$ is itself in ${\ell }^2$ only for $p>1/2$. Its preimage would be $u_j=j^{1-p}$, so the range criterion is the convergence of ${\sum }_j{j}^{2-2p}$. The P-series givesAt $p=3/2$ the reconstruction norm diverges harmonically. For $1/2<p\le 3/2$ the data are legitimate but have no preimage; for $0<p\le 1/2$ they are not even in the specified data space.

In an infinite-dimensional Hilbert space, every bounded sequence has a weakly convergent subsequence, but it need not have a norm-convergent one. For example an orthonormal sequence $e_n$ has pairwise distance $\sqrt{2}$ and therefore no strongly convergent subsequence, although $e_n\rightharpoonup 0$ by Bessel inequality.

More generally, weak sequential compactness of bounded sequences in a reflexive Banach space ensures a weakly convergent subsequence with its limit in that space. Conversely, this property for all bounded sequences characterizes reflexivity, using the Eberlein-Smulian theorem and weak compactness of the closed unit ball. Thus reflexivity replaces finite-dimensional norm compactness by weak sequential compactness.

In a nonreflexive Banach space there is no such general weak-subsequence guarantee. The coordinate unit vectors in ${\ell }^1$ give an example: by the Schur property, weak convergence would imply norm convergence, impossible for their pairwise distance two. Banach-Alaoglu theorem supplies weak-star compactness of a dual unit ball, which gives convergent subnets, not automatically convergent subsequences. For a separable predual, the weak-star compact dual ball is metrizable and subsequences do suffice. For an arbitrary Banach space one can also regard a bounded sequence in its bidual and extract a weak-star convergent subnet there; the limit need not belong to the original space. These qualifications distinguish weak from weak-star compactness.

Use the unscaled penalty convention consistent with the displayed bound:For an objective with averaged squared loss, the corresponding penalty parameter would be $\lambda /n$. We retain the unscaled $\lambda$ throughout.

By the representer theorem, the kernel ridge regression minimizer lies in the span of $k(x_i,\cdot )$. A valid coefficient vector is $\hat{a}=(K+\lambda I_n{)}^{-1}Y$, givingThe kernel matrix is a positive semidefinite matrix, so the inverse exists even if $K$ is singular. In the coefficient objective $\Vert Y-Ka{\Vert }_2^2+\lambda a^{T}Ka$, this choice satisfies $K\{(K+\lambda I_n)a-Y\}=0$; there is no need to cancel $K$. The function itself is unique because of the strictly convex squared Hilbert norm. The matrix $H$ is the kernel-ridge hat matrix in the present normalization.

Let $m^0=(f^0(x_i){)}_{i=1}^n$. Since $Y=m^0+\epsilon$, the zero mean and covariance assumption give the exact bias-variance decomposition for linear predictionNo normality of the noise is required for this identity.

Choose an orthonormal eigenbasis $v_i$ of $K$, with eigenvalues $d_i\ge 0$. Its shrinkage factors are $d_i/(d_i+\lambda )$, soThe first bound follows from $(d_i+\lambda {)}^2\ge 4d_i\lambda$ and from each shrinkage factor being at most one.

To control the bias by the Reproducing-kernel Hilbert space norm, for $d_i>0$ defineThe reproducing property gives $⟨g_i,g_{\ell }{⟩}_{\mathcal{H}}={\delta }_{i\ell }$ andIf $d_i=0$, the corresponding unscaled kernel combination has Hilbert norm squared $v_i^{T}K{v}_i=0$, so $v_i^T{m}^0=0$. There is therefore no bias component in a zero-eigenvalue direction. For the remaining directions,The scalar bound is again $(d_i+\lambda {)}^2\ge 4d_i\lambda$, and the final step is Bessel inequality. Combining the variance and bias bounds proves the prediction-risk bound for kernel ridge regression:

With exact data $f=K{u}^{†}$, the iteration error obeysThe source condition for quadratic regularization $u^{†}=K^{\ast }v$ excludes any null space component. Using its singular system of a compact operator coefficients,By Bernoulli's inequality, $(1+\tau {\sigma }^2{)}^k\ge 1+k\tau {\sigma }^2$. Also $1+k\tau {\sigma }^2\ge 2\sqrt{k\tau }\sigma$. ThereforeSumming the squared coefficients and using Bessel inequality yieldsThis is an a priori error rate from the source condition for quadratic regularization, with constant independent of $k$.

Conversely, supposeand defineFor every $N$, Bessel inequality and passage to the limit giveHence ${\sum }_i|a_i{|}^2\le C^2$. The partial sums ${\sum }_{i=1}^N{a}_i{e}_i$ are therefore Cauchy; completeness of the Hilbert space gives an element

Let $P_{N}h={\sum }_{i=1}^N⟨h,e_i⟩e_i$. For fixed $N$, coordinate convergence givesThe Hilbertian basis property gives $P_{N}h\to h$, andFirst choose $N$ large and then $n$ large. This provesfor every $h\in H$, so $x_n\rightharpoonup x_{\infty }$. This establishes the coordinate criterion for weak convergence in a separable Hilbert space.

Let $\Vert x_n\Vert \le C$. For each $i$, Bessel inequality givesso the $i$th coordinate sequence is bounded. By the Bolzano-Weierstrass theorem, choose a subsequence on which the first coordinate converges. From it choose a further subsequence on which the second coordinate converges, and continue.

Take the diagonal subsequence $(x_{\varphi (n)})$. For every fixed $i$, its $i$th coordinate converges, and it remains bounded by $C$. The coordinate criterion for weak convergence in a separable Hilbert space supplies an $x_{\infty }\in H$ such thatThis proves the theorem on a weak subsequence of a bounded Hilbert-space sequence.

No. Take the Hilbert space $H={\ell }^2$ and its standard orthonormal sequence $(e_n)$. Letthe unit sphere. This set is norm closed but is not convex. By the Bessel inequality, for every $y=(y_j)\in {\ell }^2$,so $e_n\rightharpoonup 0$; equivalently, this is a weakly null orthonormal sequence. Every $e_n$ lies in $C$, but the weak limit $0$ does not. Hence convexity is essential.

For the unscaled objective ${\sum }_i(Y_i-f(x_i){)}^2+\lambda \Vert f{\Vert }_{\mathcal{H}}^2$, the kernel-ridge hat matrix is $H=K(K+\lambda I{)}^{-1}$. Its prediction variance is ${\sigma }^2\mathrm{tr}(H^2)/n$. For eigenvalues $d_i\ge 0$, use $d_i^2/(d_i+\lambda {)}^2\le \min \{d_i/(4\lambda ),1\}$ and ${\lambda }^2{d}_i/(d_i+\lambda {)}^2\le \lambda /4$ to bound the variance and the squared bias. The latter also uses the reproducing property and Bessel inequality, and does not require invertibility of $K$.

Every orthonormal sequence in a Hilbert space converges weakly to zero, by Bessel inequality, but no subsequence converges strongly because distinct terms remain distance $\sqrt{2}$ apart.