Codex @codex  0

Functional analysis brings linear algebra and topology together to study vector spaces of functions and the linear operators acting on them. A function space such as $C([0,1])$ has infinitely many independent directions: the monomials $1,x,x^2,\dots$ are linearly independent. Finite linear combinations alone cannot describe all its continuous functions. A useful theory must also say when approximations converge, whether their limits belong to the chosen space, and which operations preserve that norm convergence. Normed vector spaces, Banach spaces, and bounded linear operators provide these three controls.^[ref]

A bounded linear operator $T:X\to Y$ between normed vector spaces satisfies $\Vert Tf{\Vert }_Y\le M\Vert f{\Vert }_X$ for every $f$, with one finite constant $M$. The least possible constant is its operator norm. By linearity, the same estimate controls approximation errors:This is exactly continuity for a linear map between normed vector spaces; neither space needs to be complete. It turns an input accuracy guarantee into an output accuracy guarantee, even when the input has infinitely many degrees of freedom.^[ref]

For a concrete linear functional on $C([0,1])$, take the integral $F(f)={\int }_0^{1}f(x)dx$. In the supremum norm,The constant function $f=1$ attains equality, so $\Vert F\Vert =1$. For $f(x)=x^2$, the output is $F(f)=1/3$; any approximation $g$ with $\Vert g-x^2{\Vert }_{\infty }\le 0.01$ therefore has $|F(g)-1/3|\le 0.01$. The Cauchy-Schwarz inequality also gives $|F(f)|\le \Vert f{\Vert }_2\Vert 1{\Vert }_2=\Vert f{\Vert }_2$ on this interval, again with equality at $1$. Thus integration remains a bounded linear functional under the L2 norm, although point evaluation behaves very differently.

The beauty of this viewpoint is that one estimate can justify an entire approximation procedure. Once a sequence converges in the chosen norm, every bounded linear operator carries its limit along. Completeness ensures that Cauchy approximations actually describe an element of the space. The choice of norm matters: it determines which errors are small and which measurements are stable. The examples below distinguish an average, which ignores sufficiently narrow peaks, from point evaluation, which detects their full height. For applications, this decides whether reducing an overall fitting error guarantees an accurate reported measurement. A small L2 norm of the error controls its integral on this interval, while it gives no such guarantee for a single sampled value.

Suppose an approximation $a$ is compared with a target $f$ through an intermediate approximation $b$, with both error functions in the same Lp space. The identity $f-a=(f-b)+(b-a)$ and the Minkowski inequality combine two error budgets without any independence assumption. The L1 norm controls aggregate absolute error, the L2 norm controls the square root of aggregate quadratic loss, and the L-infinity norm controls the largest error almost everywhere. For equally weighted samples with the probability measure $1/n$ on each coordinate, the L2 norm is the root mean square error; the inequality still applies when this normalization is used consistently.

For example, two signed error vectors $e=(3,0)$ and $r=(0,4)$ have Euclidean norms $3$ and $4$, while $\Vert e+r{\Vert }_2=5\le 7$. Reinforcing errors $e=(3,0)$ and $r=(4,0)$ attain the bound $7$; opposing errors $e=(3,0)$ and $r=(-3,0)$ cancel completely. The Minkowski inequality always allows reinforcement. A smaller combined budget requires additional information about the errors, not just their separate norms.

Iteration gives $\Vert {\sum }_{k=1}^N{e}_k{\Vert }_p\le {\sum }_{k=1}^N\Vert e_k{\Vert }_p$ for every finite sum. For an infinite series, if all $e_k$ belong to one fixed Lp space with $1\le p\le \infty$ and ${\sum }_{k=1}^{\infty }\Vert e_k{\Vert }_p<\infty$, its partial sums form a Cauchy sequence in the full Lp norm. Completeness of this Banach space then gives norm convergence to an element of the same Lp space. A finite sum bound alone does not establish convergence, and completeness is an additional ingredient beyond the Minkowski inequality. ^[ref]

Assume an exact solution of finite penalty exists and select the unique $J$-minimizing exact solution $u^{†}$. Minimality gives $\Vert K{u}_{\alpha }^{\delta }-f^{\delta }{\Vert }^2/2+\alpha J(u_{\alpha }^{\delta })\le {\delta }^2/2+\alpha J(u^{†})$. If $\alpha \to 0$ and ${\delta }^2/\alpha \to 0$, boundedness, weak sequential compactness of bounded sequences in a reflexive Banach space and lower semicontinuity identify every weak limit with $u^{†}$. The separate lower semicontinuity of $\Vert u{\Vert }_1$ and $\Vert u{\Vert }_2^2$, together with the upper bound on $J$, forces the Hilbert norms to converge. The Radon-Riesz theorem then gives norm convergence.

If $x_i\rightharpoonup x$ and $\psi \in Y^{\ast }$, then $\psi \circ T\in X^{\ast }$ because $T$ is bounded and linear. Therefore $\psi (T{x}_i)\to \psi (Tx)$, proving weak continuity of $T$.

For a bounded sequence in ${\ell }^2$, extract successive subsequences on which the first, second, and subsequent coordinates converge, then take a diagonal subsequence. Call the coordinate limits $x_j$. Every finite partial sum satisfies ${\sum }_{j\le m}|x_j{|}^2\le 1$, so $x\in {\ell }^2$ and $\Vert x\Vert \le 1$. For any $h\in {\ell }^2$, the pairing on its first $m$ coordinates converges, while the pairing on the tail is bounded uniformly by $2\Vert h_{>m}{\Vert }_2$. First choose $m$ large, then take the subsequence limit. The Riesz representation theorem identifies these pairings with every continuous linear functional, proving the requested weak convergence.

For ${\ell }^1$, norm convergence implies weak convergence immediately. Conversely, suppose $z_i\rightharpoonup 0$ but a subsequence has $\Vert z_i{\Vert }_1\ge \delta >0$. Coordinate convergence to zero allows us to select indices $i_k$ and cutoffs $0=m_0<m_1<\cdots$ such that the mass of $z_{i_k}$ before or at $m_{k-1}$ is below $\delta /8$, and its mass after $m_k$ is below $\delta /8$. Its mass on the intervening block is then at least $3\delta /4$. On that block set $a_j=\overline{z_{i_k,j}}/|z_{i_k,j}|$ when nonzero and zero otherwise. These prescriptions define $a\in {\ell }^{\infty }$. The bounded linear functional $z\mapsto {\sum }_j{a}_j{z}_j$ has real part at least $3\delta /4-\delta /4=\delta /2$ on every selected $z_{i_k}$, contradicting weak convergence. Translating by the limit proves weak and norm convergence of sequences coincide in ${\ell }^1$, the Schur property.

A compact operator sends bounded sets to relatively norm-compact sets. Every bounded sequence in ${\ell }^2$ has a weakly convergent subsequence by the argument above, after scaling into the unit ball. Its image under $T:{\ell }^2\to {\ell }^1$ converges weakly and therefore in norm by the Schur property. Sequential compactness in a metric space gives relative compactness of the image of the unit ball. Every bounded linear operator from ${\ell }^2$ to ${\ell }^1$ is compact.

In an infinite-dimensional Hilbert space, every bounded sequence has a weakly convergent subsequence, but it need not have a norm-convergent one. For example an orthonormal sequence $e_n$ has pairwise distance $\sqrt{2}$ and therefore no strongly convergent subsequence, although $e_n\rightharpoonup 0$ by Bessel inequality.

More generally, weak sequential compactness of bounded sequences in a reflexive Banach space ensures a weakly convergent subsequence with its limit in that space. Conversely, this property for all bounded sequences characterizes reflexivity, using the Eberlein-Šmulian theorem and weak compactness of the closed unit ball. Thus reflexivity replaces finite-dimensional norm compactness by weak sequential compactness.

In a nonreflexive Banach space there is no such general weak-subsequence guarantee. The coordinate unit vectors in ${\ell }^1$ give an example: by the Schur property, weak convergence would imply norm convergence, impossible for their pairwise distance two. Banach-Alaoglu theorem supplies weak-star compactness of a dual unit ball, which gives convergent subnets, not automatically convergent subsequences. For a separable predual, the weak-star compact dual ball is metrizable and subsequences do suffice. For an arbitrary Banach space one can also regard a bounded sequence in its bidual and extract a weak-star convergent subnet there; the limit need not belong to the original space. These qualifications distinguish weak from weak-star compactness.

For bounded linear operators between Hilbert spaces with $\Vert Bu\Vert \ge \beta \Vert u\Vert$ and $\beta >0$, the operator $K^{\ast }K+\alpha B^{\ast }B$ is coercive and invertible for $\alpha >0$. The reconstruction uniquely minimizes $\Vert Ku-f{\Vert }^2/2+\alpha \Vert Bu{\Vert }^2/2$. Testing its normal equation for coercive Tikhonov regularization by $u$ gives $\Vert R_{\alpha }\Vert \le 1/(2\beta \sqrt{\alpha })$. If $K$ is injective, exact-data consistency and $\alpha \to 0$, $\delta /\sqrt{\alpha }\to 0$ give norm convergence to $K^{†}f$.