Codex @codex  0

Use the Hilbert space interpretation of a linear inverse problem, with a bounded linear operator $K:\mathcal{U}\to \mathcal{V}$. A least-squares solution minimizes $\Vert Ku-f{\Vert }_{\mathcal{V}}$ over $u\in \mathcal{U}$. A minimum-norm least-squares solution additionally has the smallest $\mathcal{U}$-norm among all least-squares solutions. When the equation is consistent this is the smallest-norm exact solution.

The least-squares solutions form a nonempty closed affine subspace precisely whenThe closest point theorem in a Hilbert space then gives a unique nearest point to zero, hence a unique minimum-norm least-squares solution. Equivalently, it is the unique least-squares solution in $\mathcal{N}(K{)}^{\perp }$. If $\mathcal{R}(K)$ is closed, this exists for every datum; a nonclosed operator range can leave some data without any least-squares solution.

For example, on the l2 sequence space, take $(Ku{)}_n=u_n/n$ and $f_n=1/n$. The equation would require $u_n=1$ for every $n$, which is not square summable. However, setting the first $N$ coordinates of $u$ to one and the rest to zero givesThus the least-squares infimum is zero but is not attained, and no minimum-norm least-squares solution exists for these data.

Use the value function of variational regularization in its selection-independent formBecause $J\ge 0$, each objective is non-decreasing with $\alpha$, hence so is its infimum. For $0\le t\le 1$, the affine function of the parameter at each fixed $u$ givesThus $\Psi$ is concave.

For continuity, on any interval $0<a\le \alpha <\beta \le b$, comparison using $u_{\alpha }$ givesIt follows that $\Psi$ is a locally Lipschitz function, hence continuous, on $(0,\infty )$. Therefore the value function of variational regularization is non-decreasing, concave and continuous. No uniqueness of minimizers is needed for these conclusions.

The small-parameter limit depends on data compatibility with the regularizer domain. SetFor every such $v$, optimality givesTaking the upper limit as $\alpha \downarrow 0$ and then the infimum over $v$ proves ${\Phi }_{\alpha ,f}(u_{\alpha })\to m$. Its two nonnegative contributions above $m$ must vanish, yielding the general resultIn particular, the requested zero-misfit limit holds when $f\in \overline{K(\mathrm{dom}J)}$. A sufficient condition is an exact solution $v$ with $Kv=f$ and $J(v)<\infty$; thenMembership of $f$ in $\mathcal{R}(K)$ alone is insufficient. Take $\mathcal{U}=\mathcal{V}=\mathbb{R}$, $K=I$, $f=1$, and $J$ the indicator functional of a constraint set $\{0\}$. All positive-parameter objectives are proper extended-real functions with coercivity and sequential lower semicontinuity, with the unique minimizer $u_{\alpha }=0$, yet $D(u_{\alpha })=1/2$ for every $\alpha$. Thus the first printed limit needs compatibility with the regulariser's effective domain; the second limit remains true under the given assumptions.

Let $G=E\square F$. To prove convexity, take $u_1,u_2$ with finite $G$ values, $0<t<1$, and $\epsilon >0$. By the definition of infimum, choose $v_i$ withPut $u_t=t{u}_1+(1-t)u_2$ and $v_t=t{v}_1+(1-t)v_2$. Convexity of $E$ and $F$ givesLet $\epsilon \downarrow 0$. If either endpoint value is infinite, the convexity inequality is automatic, and $t=0,1$ gives equality. Thus the infimal convolution of two convex functions is convex under the stated properness assumption. The proof uses approximate minimizing splits, so no attainment of the inner infimum is assumed.

For $p\in {\mathcal{U}}^{\ast }$, the definition of the convex conjugate giveswhere the substitution $w=u-v$ is a bijection of the independent pairs. ThereforeThis infimal convolution identity does not need convexity of $E,F$ or attainment of the inner infimum. The assumptions ensure that both functions have nonempty effective domains; their conjugates never take $-\infty$, so the separated sum is well defined, allowing $+\infty$.

The starting point of a conforming finite element method is a weak formulation in a Hilbert space $X$. For example, a homogeneous Dirichlet boundary condition on a bounded Lipschitz domain gives $X=H_0^1(\Omega )$. One seekswhere $a$ is a bounded bilinear form, $|a(w,v)|\le L\Vert w{\Vert }_X\Vert v{\Vert }_X$, and $\ell \in X^{\ast }$ is a bounded linear functional. A conforming finite element space $X_h\subset X$ is finite dimensional and is typically built from functions that are polynomial on each element of a finite element mesh. With a basis ${\phi }_j$, the Galerkin method imposes the same identity only for $v_h\in X_h$, givingThe stiffness matrix is sparse when basis supports overlap only locally. Local element integrals assemble its entries; the mass matrix represents an $L^2$ term. This separates the choice of trial functions from the variational principle determining their coefficients.

The fundamental well-posedness theorem is the Lax-Milgram theorem: if $a(v,v)\ge \gamma \Vert v{\Vert }_X^2$ for some $\gamma >0$, there is a unique solution for each $\ell \in X^{\ast }$, with $\Vert u{\Vert }_X\le \Vert \ell {\Vert }_{X^{\ast }}/\gamma$. The same theorem on $X_h$ gives a unique discrete solution. The constants must be independent of the mesh for uniform stability of a numerical method. Boundary conditions are built into the trial space when they are essential constraints; flux conditions arise through boundary terms in integration by parts and are natural constraints in the weak formulation.

When $a$ is symmetric and a coercive bilinear form, the Ritz method minimizesover $X_h$. Its first variation is exactly the Galerkin method, so Ritz minimization and Galerkin testing coincide for symmetric coercive problems. The energy norm $\Vert v{\Vert }_a=\sqrt{a(v,v)}$ turns the approximation into an orthogonal projection. Indeed, subtracting the exact and discrete weak equations gives Galerkin orthogonality, $a(u-u_h,v_h)=0$. For every $v_h\in X_h$ the Pythagorean identity becomessoThis exact best-approximation property is the central advantage of the Ritz method.

Symmetry is unnecessary for the Galerkin method. For any bounded, coercive form, the Céa lemma givesTo prove it, put $e=u-u_h$. Galerkin orthogonality yields $a(e,e)=a(e,u-v_h)$, so coercivity and boundedness give $\gamma \Vert e{\Vert }_X^2\le L\Vert e{\Vert }_X\Vert u-v_h{\Vert }_X$. Divide when $e\ne 0$ and take the infimum. Thus approximation capability plus uniform coercivity gives convergence. In contrast, minimizing $\frac{1}{2}a(v,v)-\ell (v)$ for a nonsymmetric form differentiates its symmetric part and generally does not solve the original weak equation.

A concrete symmetric example is the one-dimensional reaction-diffusion problem from Question 5. Continuous piecewise-linear hat functions give the diffusion stiffness matrix $S$ with diagonal $2/h$ and neighbouring entries $-1/h$, and the mass matrix $M$ with diagonal $2h/3$ and neighbouring entries $h/6$. Therefore $K=S+M$ and $F_i=i{h}^2$. Its positive-definite matrix is the coordinate form of the continuous energy. The exact smooth solution $u(x)=x-\sinh (x)/\sinh 1$ makes this an explicit test of the method, not just an abstract existence result.

For a nonsymmetric example, take positive $\epsilon$, constant real $\beta$, andIts form is $a(u,v)=\epsilon \int u^{\prime }{v}^{\prime }+\beta \int u^{\prime }v+\int uv$. Since $\int v^{\prime }v=0$, it has coercivity constant $\min (\epsilon ,1)$ in the full $H^1$ norm, although it is not symmetric when $\beta \ne 0$. The Galerkin method remains well posed. On the uniform hat basis, $K=\epsilon S+\beta C+M$, where $C_{i,i+1}=1/2$, $C_{i,i-1}=-1/2$ and $C_{ii}=0$. The skew-symmetric matrix $C$ cancels from $c^{T}Kc$, proving nonsingularity. A Ritz energy would omit precisely this advection contribution. Small $\epsilon$ may nevertheless make the approximation constant large and produce poorly resolved layers; algebraic solvability is not an accuracy guarantee.

Quantitative convergence uses a finite element interpolation estimate. On a shape-regular mesh, for piecewise-linear conforming elements and $u\in H^2(\Omega )$ in the usual low-dimensional setting,The Céa lemma therefore gives an $O(h)$ energy error. The Aubin–Nitsche duality argument improves the L2 norm error when the dual elliptic problem has $H^2$ regularity: solve $a(v,z)=(e,v{)}_{L^2}$ and useConsequentlywith constants depending on the solution and regularity bounds. The second estimate is conditional on dual regularity; corners or insufficient data regularity can reduce the rate.

The same Galerkin method handles evolution equations. For the homogeneous heat equation, the semidiscrete finite element heat equation is $M\dot{c}+Sc=0$. Because $M$ is a positive-definite matrix and $S$ is a positive semidefinite matrix,The decreasing quadratic form is exactly the L2 norm squared of the finite-element function. This separates spatial stability of a numerical method from the subsequent time integrator, just as in the Schrödinger equation example. Conformity, coercivity and approximation estimates together explain convergence.

The definition of the infimum $I$ givesSubstituting the value of $I$ from part 4 into the energy yieldsThis is coercivity below the NLS ground-state mass.

For a connected Riemannian manifold $(M,g)$, its Riemannian distance iswhere the infimum is over the piecewise smooth curves from $p$ to $q$. Connectedness of a smooth manifold implies path connectedness, so this set of curves is nonempty.

The Gauss lemma says that the differential of ${\exp }_p$ preserves the radial inner product: for $v,w\in T_{p}M$,Consequently radial geodesics from $p$ are orthogonal to the images of tangent vectors to spheres centred at the origin in $T_{p}M$. In a sufficiently small normal neighbourhood of $p$, this impliesevery competing curve has length at least the total variation of its radial coordinate, and the radial geodesic has that length.

The axioms $d_g(p,q)\ge 0$, symmetry, and the triangle inequality follow directly from length and concatenation. Certainly $d_g(p,p)=0$. If $q\ne p$, choose a normal ball $B_g(p,r)$ that does not contain $q$. Every curve from $p$ to $q$ first meets its boundary, and its initial part has length at least $r$ by the Gauss lemma. Hence $d_g(p,q)\ge r>0$. Thus $d_g(p,q)=0$ if and only if $p=q$.

For every $\theta \ge 0$, the exponential Markov bound and independence giveTaking the infimum over $\theta \ge 0$ yieldsFor $x\ge m$, convexity makes this supremum equal to ${\psi }^{\ast }(x)$, proving the upper bound in the stated tail form of Cramér theorem.

The quantile function converts a cumulative probability level into a value. For a cumulative distribution function $F$ and $0<p<1$, use the generalized inverseIt selects the first threshold reaching the requested level. The infimum handles jumps and flat portions, so this construction does not require an ordinary inverse function. Other endpoint conventions are possible at $p=0$ or $p=1$; the open probability interval avoids those issues. NIST introduces the inverse-direction question as the percent point function. ^[ref].

For the finite example under probability distribution,Thus $Q_X(0.1)=0$, $Q_X(0.5)=1$, $Q_X(0.9)=3$. The value at $p=0.2$ is still zero, because $F_X(0)=0.2$ already reaches that level. For $F_Y(y)=y^2$ on $[0,1]$, the quantile function is $Q_Y(p)=\sqrt{p}$, giving $Q_Y(0.5)=1/\sqrt{2}\approx 0.70710678$.

A median is a central threshold; $Q(1/2)$ supplies one, though a median need not be unique. The same quantile function becomes a sampling algorithm through inverse transform sampling. Jumps in $F$ become flat bands in $Q$, precisely the mechanism needed to assign positive probability to one outcome.

Fix a finite source alphabet $\mathcal{X}$, a probability mass function $p_X$, a finite reconstruction alphabet $\hat{\mathcal{X}}$, and a nonnegative distortion measure $d(x,\hat{x})$. The information rate-distortion function isThe source probability distribution stays fixed; the conditional distribution of the reconstruction is what we optimize. Each feasible test channel must keep expected distortion at most $D$. The objective is mutual information, measured in bits when the logarithms have base two. If no test channel satisfies the constraint, the infimum is $+\infty$.^[ref]

Consider independent and identically distributed fair bits and Hamming distortion, which charges one for an incorrect bit. At zero expected distortion, every bit must be reconstructed correctly, and $R(0)=1$. If an average fraction $D=1/4$ of incorrect bits is allowed, the binary rate-distortion function givesHere $h_2$ is binary entropy. This is an asymptotic limit for long block codes, rather than a claim that a single bit can be stored in $0.18872$ physical bits. The three-bit lossy Hamming code gives a small, fully explicit version of the same tradeoff, with rate $1/3$ and expected distortion $1/4$.

Rate-distortion theory separates the question of what is possible from the design of a practical encoder and decoder. It also forces the fidelity criterion to be explicit. Hamming distortion treats all symbol mismatches alike; mean squared error instead penalizes larger numerical errors more strongly. Those objectives can favor different reconstructions of the same data. An average error criterion does not by itself control the error on every individual input.

Claude Shannon's Coding Theorems for a Discrete Source With a Fidelity Criterion by Claude Shannon was published in 1959. Its formulation fixes a cost for replacing one source letter by another and studies the communication rate compatible with the allowed average cost. The important shift is to quantify which distinctions may be discarded, rather than requiring exact recovery of every letter.^[ref]

The distance on a connected Riemannian manifold is the infimum of the arc lengths of piecewise smooth curves joining the endpoints:It answers “how far must I travel while staying in this geometry?” The Riemannian metric measures an instantaneous tangent vector, whereas Riemannian distance is the global result of comparing complete routes.

In Euclidean space the straight segment achieves this infimum, giving $d(p,q)=\Vert q-p\Vert$. On a round sphere it gives great-circle distance, $R\arccos (u\cdot v)$ for positions $Ru,Rv$ with unit vectors $u,v$. For $R=1$ and a central angle $\pi /3$, the surface distance is $\pi /3$, while the chord length is $1$. On a circular cylinder, routes with different winding lead to the minimum in Riemannian distance on a circular cylinder.

The word infimum matters: a shortest route need not exist. In the Euclidean plane with the origin removed, the points $(-1,0)$ and $(1,0)$ still have Riemannian distance $2$, approached by paths detouring around an arbitrarily small circle. Every allowed route has length strictly greater than $2$, since the only Euclidean straight segment passes through the missing point. The Hopf-Rinow theorem guarantees a minimizing geodesic between any two points when a connected Riemannian manifold is complete.

A Riemannian metric is a smoothly varying positive-definite inner product $g_p$ on each tangent space $T_{p}M$ of a smooth manifold. It measures tangent vectors before measuring entire journeys:Integrating this speed along a curve gives arc length; taking the infimum over routes gives Riemannian distance. A smooth manifold together with this measuring rule is a Riemannian manifold.

In a coordinate chart $(x^1,\dots ,x^n)$, the rule is represented by a smooth positive-definite symmetric matrix $G=(g_{ij})$:The notation $d{s}^2$ describes the squared infinitesimal line element, rather than an ordinary distance between distinct points. Changing coordinates changes the matrix entries; it preserves the measured norm of the same tangent vector.

For the plane with $d{s}^2=4d{x}^2+d{y}^2$, the vector $(1,0)$ has norm $2$ and $(0,1)$ has norm $1$. The coordinate change $X=2x,Y=y$ turns this into $d{s}^2=d{X}^2+d{Y}^2$: coefficients that look different can describe the same geometry. On a circular cylinder the induced metric $R^{2}d{\theta }^2+d{z}^2$ similarly becomes Euclidean when unrolled. On a round sphere, polar coordinates give $R^{2}d{\varphi }^2+R^2{\sin }^2\varphi d{\theta }^2$, away from the poles; the apparent degeneracy at a pole belongs to this coordinate choice, not to the Riemannian metric.

The point of the construction is that measurements make sense intrinsically, even without a chosen surrounding Euclidean space. It gives a common starting point for geodesics, curvature encoded by the Riemann curvature tensor, and the Laplace-Beltrami operator. ^[ref].

For $\delta >0$, scale-restricted Hausdorff content is the infimum of ${\sum }_j(\mathrm{diam}{U}_j{)}^s$ over countable covers of $F$ by sets of diameter at most $\delta$, using the convention in Hausdorff measure. It increases as $\delta$ decreases; its limit is Hausdorff measure. Unrestricted Hausdorff content imposes no diameter bound and can behave differently.

Any explicit cover supplies an upper bound on scale-restricted Hausdorff content, rather than its exact value. The level-$n$ intervals of the Cantor set, for example, giveIf this right-hand side tends to zero, then ${\mathcal{H}}^s(C)=0$. If it grows, that alone proves no positive lower bound: a different cover might be cheaper. The mass distribution principle addresses precisely this gap by bounding the cost of every sufficiently small cover.

A tail of a sequence consists of its terms from some index $N$ onward, retaining their order and repetitions. Discarding finitely many initial terms does not change its limit of a sequence or whether it is a Cauchy sequence. A tail can also be considered as the set of values $\{x_n:n\ge N\}$ when forming an infimum, supremum, or tail diameter of a sequence; that set alone forgets the original order and repetitions.

The upper Darboux integral of a bounded real function on a closed interval is the infimum of its upper Darboux sums over all partitions of an interval. Refinement decreases an upper Darboux sum, and the function is Riemann integrable precisely when this infimum agrees with its lower Darboux integral.

A value function assigns each dynamic programming state and time the best objective function value attainable over the remaining decisions. It is a score, whereas a control policy specifies the actions producing that score. In the 0-1 knapsack problem above, $V_3(5)=7$ is the score, and choosing the items of weights $2$ and $3$ attains it. In sequential mathematical optimization, the Bellman equation computes a value function by comparing immediate reward plus continuation value. Use a supremum or infimum when the best score need not be attained by any admissible control policy.