For a bounded linear operator between Hilbert spaces, a least-squares solution of data is a minimizer of the residual norm:
The minimum-norm least-squares solution is the one of smallest -norm among these minimizers. Equivalently it is the unique least-squares solution in . If the equation is consistent, this is the smallest-norm exact solution.
Let be the orthogonal projection onto . The orthogonal decomposition by a closed subspace gives
The first term has infimum zero by the definition of closure; that infimum is attained exactly if . This proves the least-squares existence criterion:
They exist for every datum precisely when the range is closed. Once a solution exists, all solutions differ by elements of the closed null space, and projecting onto its orthogonal complement gives the unique minimum-norm solution.
For a counterexample to existence, take , , and . The range is dense, so the least residual infimum is zero; finite truncations realize residuals tending to zero. But an exact preimage would have every coordinate equal to one and would not belong to . Thus no least-squares minimizer exists for this datum.
Let denote the orthogonal complement of inside . The Meyer-Mallat theorem constructs a wavelet whose integer function translations form an orthonormal basis of . Dilation then gives the orthogonal decomposition into orthogonal complements , so its translates and dilates form an orthonormal wavelet basis.
Using the MRA low-pass filter, choose the high-pass symbol . The quadrature mirror filter identity makes the two analysis channels orthonormal. The resulting Fourier transform formula is
Equivalently, with the refinement coefficients from the scaling refinement equation, put and . This index choice gives precisely the displayed high-pass symbol. Changing all by one constant unit phase gives the same orthonormal wavelet construction. In particular the often-used convention differs only by a global minus sign. The basis is for .
The orthogonal projections of the Gaussian noise onto the column space of and its orthogonal complement are independent. Consequently and the residual sum of squares are independent, and
Thus under the alternative,
This is the noncentral t-distribution. The rejection region is fixed by the design, and the chosen level, while this probability distribution depends on only through . Therefore the statistical power does not depend on the other regression coefficients, with the design and fixed.
Let be the smooth nonzero tangent along a regular null curve, and . Metric compatibility gives . In a two-dimensional Lorentzian manifold, the orthogonal complement of a nonzero null vector is its own span; this follows directly in the preceding null basis. Therefore .
Consequently every regular null curve is a geodesic up to reparametrization. To obtain an affine parameter , write and solve . Then . A solution exists locally and along any nonsingular parameter interval. The original parameter need not be affine; the literal assertion of zero covariant acceleration for every parameter would be false.
For the Petersen graph, each vertex has three neighbours, adjacent vertices have no common neighbour, and distinct nonadjacent vertices have exactly one common neighbour. The entries of count length-two walks, so diagonal entries are three, adjacent entries zero, and other off-diagonal entries one. Hence
The connected graph has the simple eigenvalue three with eigenvector . On its orthogonal complement , so the remaining eigenvalues satisfy and are one or minus two. If their multiplicities are , then and . Thus
Put . Then and , with the inner product linear in its second argument as required by the later SVD formula. For nonreal , self-adjointness gives . For , positivity gives . These bounds give injectivity and closed range. The range is dense because its orthogonal complement is the kernel of the adjoint, to which the same bound applies. Thus the range is all of and the inverse is bounded in either case. Consequently
If is compact, composition of the compact operator with bounded is compact, so is compact too.
The vector triple product identity gives , and hence
Thus is the identity on the orthogonal complement and multiplies the component parallel to by . It is invertible exactly when , and then
A self-adjoint operator satisfies . If , then , so . An eigenvector exists over ; its orthogonal complement is -invariant. Induction on dimension produces an orthonormal basis of eigenvectors, proving the finite-dimensional spectral theorem.
A self-adjoint is positive definite exactly when every eigenvalue is positive, since in an eigenbasis. The quadratic forms of two positive-definite operators add, so their sum is positive definite.
The same eigenbasis shows that exactly when is positive definite for every and negative definite for every . Finally, if is a unit eigenvector of with eigenvalue , the Rayleigh quotient bounds give
Thus every eigenvalue of lies in .
The Lax-Milgram theorem assumes that is a real Hilbert space and is a bounded bilinear form and a coercive bilinear form: there are such that
Then for every continuous linear functional there is a unique satisfying
Symmetry of is not required.
For the proof, the Riesz representation theorem provides a bounded linear operator and with and . Coercivity and the Cauchy-Schwarz inequality imply
Thus is injective and its range is closed: if converges, the displayed inequality makes a Cauchy sequence; completeness gives and . If lies in the orthogonal complement of the range, then for all . Taking gives , so . A closed subspace with zero orthogonal complement is all of , hence is surjective. The unique solution is , and the same inequality gives the asserted norm bound.
At a point , write and take the orthogonal complement with respect to the Riemannian metric . The compatible triple identity gives
If is a Lagrangian submanifold, then for , so . Both have dimension , hence equality.
Conversely, if , equality of dimensions gives , and for gives . Therefore
A positive-definite inner product has . Hence , proving that every Lagrangian submanifold is a totally real submanifold for a compatible almost complex structure.
The map is a bounded linear operator, so the inverse image of the closed singleton is closed. That inverse image is also a convex set: if , every convex combination has the same image.
The identity gives
Consequently such a exists exactly when is the sum of an element of and one of its orthogonal complement. Thus
These are precisely the least-squares solutions. Indeed, if and , then for every the Pythagorean identity gives
Conversely, differentiating the squared residual along any real direction forces , hence the normal equation for a linear inverse problem. For any , the difference of two solutions of the normal equation for a linear inverse problem lies in , since . Therefore
a closed affine subspace; the empty case is closed and convex as well.
Either essay option suffices; both are developed here to make the two mathematical constructions available.
Diffusion for images. Model grey level as , initially , and use a Neumann boundary condition to avoid flux across an image boundary. The linear heat equation smooths the data. On its solution is convolution with the heat kernel, . Equivalently each Fourier mode is multiplied by , suppressing high frequencies and noise. With no flux, the mean is conserved and
The drawback is that sharp edges also contain high frequencies: a step becomes a transition of width comparable to . Running the heat equation backwards attempts sharpening but amplifies modes by and is ill-posed.
The Perona-Malik equation instead uses a decreasing diffusivity,
Small gradients are smoothed strongly and large ones less strongly. If and is a unit tangent to a level curve, then away from zero gradient,
The derivative of the flux, rather than just , controls forward parabolicity. Here becomes negative for : smoothing remains tangential but the normal direction can sharpen. This formal edge enhancement comes with forward-backward ill-posedness, so existence and stability of the unregularized continuum equation must not be presumed. One regularization uses as a smoothed edge detector while keeping the flux proportional to . For fixed positive , the coefficient is controlled by the smoothed data; under suitable bounds it stays positive and gives a regularized forward equation. It preserves edges through reduced cross-edge transport, without the same local backward-diffusion calculation.
Diffusion from a gradient energy connects these equations with variational regularization. If , its formal gradient flow is . Linear diffusion corresponds to . The displayed Perona-Malik equation corresponds to , which is nonconvex in the gradient for large . A convex alternative is total variation flow, with , interpreted through a subgradient at zero gradient. Adding squared data fidelity gives the formal evolution , whose equilibrium is the unique total variation denoising minimizer. Convex TV preserves sharp interfaces more effectively than the heat energy, but it may produce piecewise constant plateaux, known as staircasing in total variation denoising. A stopping time or fidelity weight controls the smoothing scale. Linear diffusion is stable but blurs edges; nonlinear diffusion must balance edge selectivity with parabolicity and regularization.
Wavelets on an interval. Simply restricting a whole-line orthonormal wavelet to destroys its orthogonality and generally its vanishing moments. Periodizing the basis restores orthogonality on the circle, but treats the two endpoints as neighbors. This is suitable for periodic data and can create an artificial endpoint jump for nonperiodic data.
A localized interval-adapted wavelet basis instead uses unchanged interior functions and finitely many special boundary wavelets at each endpoint. Start with a sufficiently regular compact support orthonormal Daubechies wavelet of order at least and choose a coarse level at which left and right boundary supports are disjoint. At each boundary, take appropriate finite combinations of the scaling functions that meet the endpoint, restricted to the interval. Choose these combinations to reproduce polynomials of degrees , remove dependencies, and orthonormalize the finite boundary Gram matrix. The choices must be compatible with refinement so that the resulting finite-dimensional spaces are nested; independent arbitrary orthonormalizations would not ensure this. Interior functions retain their whole-line filters, while the boundary functions use finite boundary refinement matrices.
For each level, choose an orthonormal basis of the orthogonal complement . It consists of interior wavelets and a bounded number of left and right boundary wavelets. Since the polynomial restrictions of degree less than belong to , every member of has vanishing moments. The compatible local boundary construction retains support diameter and the regularity of the interior construction. Boundary modification affects only finitely many functions at each level, so increasing resolution still makes the union dense in . Consequently, for a fixed coarse level ,
The coarse scaling functions together with all these wavelets form an orthonormal basis. This construction is the Cohen-Daubechies-Vial interval wavelet construction. It preserves localization, polynomial cancellation and stable coefficient extraction without imposing periodic or zero boundary data. The finite boundary refinement matrices also permit a fast transform; a mere restriction followed by one unrelated Gram-Schmidt process at each scale does not establish all these properties.
An eigenvalue of a matrix is a scalar for which for some nonzero eigenvector . Its corresponding eigenspace is
Write . The displayed matrix is the outer product , so
Its column space is contained in and is nonzero because . Hence is a rank-one matrix.
The vector is an eigenvector with eigenvalue , and every vector in the orthogonal complement has eigenvalue . Thus
Since is a direct sum of these eigenspaces, has an eigenbasis and is therefore a diagonalizable.
The usual sufficient input promises for the HHL algorithm are efficient quantum state preparation of the nonzero vector , efficient access to a sparse matrix with at most nonzero entries per row, and a condition number after an efficiently known normalization. Both the positions and values of the nonzero matrix elements must be computable coherently in polynomial time. More generally, efficient Hamiltonian simulation can replace the sparsity promise. We assume is invertible; if zero eigenvalues occur, one must instead restrict to their orthogonal complement and specify the desired inverse on that support.
With and a known bound , choose . The HHL controlled reciprocal rotation then succeeds with probability
This is the required inverse-polynomial lower bound. In the usual approximate algorithm, inverse-polynomial requested error also gives polynomial runtime under these input promises. The original HHL paper states the dependence on sparsity, conditioning, and accuracy.
There is a qualification for a literally exact version. Representability of the eigenvalues in bits does not itself make exact quantum phase estimation efficient: using requires controlled powers corresponding to evolution times as large as . Under the paper's idealization we may describe their exact action, but polynomial runtime for that exact circuit additionally requires efficient implementations of those controlled powers, or equivalent efficient exact spectral access. Assuming an operation executes exactly does not bound its cost. This distinction is recorded in cost of exact phase estimation on a dyadic spectrum.
Consider a bounded Lipschitz domain and homogeneous Dirichlet boundary conditions for
Assume is a bounded real symmetric matrix field with uniformly, is bounded, and defines a bounded linear functional on , the zero-boundary Sobolev space. The weak formulation is
The Cauchy-Schwarz inequality gives continuity . Uniform ellipticity and the Poincaré inequality give a coercive bilinear form, . The Lax-Milgram theorem therefore supplies a unique weak solution, with . Its proof represents by a bounded linear operator using the Riesz representation theorem; coercivity gives an injective operator with closed image, and a zero orthogonal complement makes that image all of . This yields existence, uniqueness and the displayed estimate.
For a conforming finite element space , the Galerkin method finds such that for every . With a basis , this is the linear system , where the stiffness matrix is and . Its positive-definite matrix property proves unique discrete solvability. Local support gives a sparse matrix through elementwise assembly.
For the symmetric problem, the Ritz method minimizes over . Differentiating in every direction gives precisely the Galerkin method equations. Conversely, if solves them, , proving it is the unique minimum. The same argument over gives , where is the energy norm. Thus the Ritz method chooses the best trial function in that norm.
Subtracting the continuous and discrete equations gives Galerkin orthogonality, . For arbitrary and , it follows that . Coercivity and continuity prove the Céa lemma:
In the symmetric energy norm, the Cauchy-Schwarz inequality improves the constant to one. Equivalently, Galerkin orthogonality gives the Pythagorean identity . These estimates establish stability and show that approximation properties of the trial space determine convergence.
For continuous piecewise-linear elements on a shape-regular mesh in dimensions at most three, a finite element interpolation estimate gives when . Combining it with the Céa lemma yields energy norm error. More generally, degree- elements give error in under the corresponding regularity and approximation assumptions. Mere mesh refinement cannot supply an order whose required solution regularity is absent.
An additional order in the L2 norm follows under a dual elliptic regularity assumption. Solve and assume . The Aubin–Nitsche duality argument uses Galerkin orthogonality to obtain
Therefore , giving for the piecewise-linear case with . This improvement explicitly requires regularity of the dual problem.
As a concrete example, solve on with zero endpoint values. On a uniform mesh of spacing , the piecewise-linear hat functions give , and . The discrete equations are , whose solution is . The exact solution is , so is its piecewise-linear nodal interpolant. On each interval ,
Integrating the squared error and squared derivative error gives the explicit rates
where the norm here is .
For nonsymmetric coercive problems, the Galerkin method, Galerkin orthogonality and the Céa lemma still apply, but the symmetric quadratic minimization interpretation of the Ritz method generally does not. Nonconforming trial spaces, inexact integration and noncoercive equations need additional arguments beyond the conforming coercive theory proved here.
Use the shape-rate convention for the Gamma distribution. Define
The zero null space assumption makes a positive-definite matrix, and is the unique ordinary least squares solution. The Gaussian likelihood contributes , which must be retained because is unknown. Multiplying it by the improper prior and the Gamma distribution gives the normal-gamma posterior with a flat prior:
Here the constant density of the improper prior cancels on normalization. Put
The normal equation makes the residual orthogonal to the range of . The orthogonal decomposition by a closed subspace into that range and its orthogonal complement gives
Integration over by the Gaussian integral, followed by integration over using the Gamma distribution, yields
Thus the posterior distribution is proper, despite the improper prior; here follows from the zero null space assumption.
Completing the square in and collecting the powers of give the two conditional distributions:
Explicitly their probability density functions are
For a joint maximum a posteriori estimate relative to , first fix . The unique maximizing is . The remaining log density is plus a constant. Consequently, if ,
The stated assumptions do not always imply . If , the supremum occurs as but is not attained on ; if , the density is unbounded there. In either case there is no joint maximum a posteriori estimate in the stated parameter space. Calling zero a boundary mode does not make it an admissible positive precision.
If “MAP estimators” instead means separate modes of the marginal distributions, integration gives
Thus the marginal maximum a posteriori estimate of is always , and that of is when , with the same nonattainment at zero when . Joint and marginal maximization are different operations; the usual joint interpretation gives the preceding boxed pair.
Here , so the Jacobi iteration matrix is
The all-ones vector is an eigenvector with eigenvalue . On its two-dimensional orthogonal complement, the coordinates sum to zero and , so the other eigenvalue is with multiplicity two. Therefore
By Jacobi convergence for a three-by-three equicorrelation matrix, convergence occurs exactly for
At either endpoint the spectral radius is one, so convergence for arbitrary initial data fails.
Write . We first prove that is dense. Otherwise some nonzero lies in its orthogonal complement, so
By part (b), choose finite-dimensional spaces containing such that in operator norm. Since
the operator is not injective. Therefore is not surjective; the contrapositive of part (c) makes an eigenvalue of . Choose unit vectors with . Then
Compactness gives a subsequence for which converges. Since , the displayed relation makes converge to a unit vector , and continuity gives , contradicting the hypothesis. Thus is dense.
Next suppose were not bounded below, or equivalently that its lower norm were zero. There would be unit vectors with , hence
Again a convergent subsequence of forces the corresponding to converge to a unit vector satisfying , another contradiction. Therefore some satisfies
This lower bound makes closed: if converges, then is Cauchy and its limit maps to the same limit. Since the image is both dense and closed, it is all of . We have proved the Fredholm alternative for a compact operator: every nonzero spectral value of a compact operator is an eigenvalue.
Write . Since is a unit vector, the equation becomes . Thus ranges over the two-dimensional orthogonal complement of : the locus is an affine hyperplane through , with unit normal . Conversely, every such vector satisfies the equation. Its perpendicular distance from the origin is ; is the signed distance in the chosen normal direction.
Write with . The orthonormality conditions and give
Consequently . It is the orthogonal projection matrix onto the orthogonal complement of the linear span of .