Put . The image penalty is discrete isotropic total variation ; the fidelity term is the L1 norm . Introduce scalars and minimize the linear objective
subject to the following affine residuals being cone-feasible:
where is the Lorentz cone. Every component displayed is affine in , so stacking them gives exactly the requested form, with
The first two inequalities imply , so separate nonnegativity constraints for are unnecessary. The cone constraints imply . Every lifted feasible objective bounds the original objective from above; choosing and gives equality. Since both objective weights are positive, an optimum uses these tight values. Thus the lift is an exact second-order cone program for box-constrained TV-L1 denoising.
The product cone is proper, closed, convex and self-dual. The stacked has full column rank: the box rows recover , the fidelity rows then recover , and the first coordinates of the Lorentz blocks recover . Strict feasibility is also explicit: choose , then take and .
For generic cone slack , a canonical barrier is
on and . The positive sheet condition matters: positivity of the quadratic expression alone would also include the wrong Lorentz nappe.
Composing the barrier with the affine residual map gives
Its domain includes the strict inequalities above. Each orthant coordinate contributes one to the self-concordant barrier parameter and each Lorentz block contributes two. Therefore the product-cone logarithmically homogeneous barrier has
The affine composition is the barrier used in the primal variables; it need not itself be logarithmically homogeneous in because the image data and box offsets are affine constants.
The Szemerédi theorem says that for every integer and every , there is such that, for , every with contains a nonconstant arithmetic progression
Equivalently, every subset of the positive integers with positive upper asymptotic density contains arithmetic progressions of every finite length. Length one is immediate.
The Furstenberg multiple recurrence theorem states that for every probability measure-preserving system, every measurable with , and every integer , there is with
Here means a preimage, so invertibility is unnecessary. The Furstenberg multiple recurrence theorem also does not assume an ergodic transformation.
We prove the implication to the finite Szemerédi theorem by constructing the Furstenberg correspondence principle explicitly. Suppose, to the contrary, that for fixed and there are and with , but with no length- arithmetic progression of positive common difference. In the binary full shift , encode as , taking all coordinates outside to be zero. Let be the left shift, , and set
The cylinder set is a clopen set, and . The binary full shift is a compact metric space, so compactness of probability measures on a compact metric space gives a subsequence of these empirical measures with weak convergence of probability measures to a Borel probability measure .
For every continuous on the full shift,
Passing to the weak limit shows that is an invariant measure for the continuous left shift. Thus is a probability measure-preserving system. Since the indicator function of is continuous, .
Apply the Furstenberg multiple recurrence theorem to . For some , the clopen set
has . Its continuous indicator function gives . For large , at least one in the defining empirical measure therefore belongs to . The choice of left shift implies
so lies in . This contradicts the assumed absence of arithmetic progressions and proves the finite Szemerédi theorem. Applying the finite Szemerédi theorem on intervals where an infinite set has density bounded below proves the stated upper asymptotic density formulation.
For multiple recurrence for circle rotations, write and , with normalized Lebesgue measure , which is the Haar measure of the circle group. Fix a measurable with and . If is rational, there is with the identity, and gives an intersection of measure .
For irrational , the pigeonhole principle applied to in equal arcs gives with . In particular there are arbitrarily small nonzero returns to zero for the irrational rotation of the circle.
We also need translation continuity in L1 on the circle. Given a measurable , approximate its indicator function in the L1 norm by a continuous on the circle group. Such approximation follows from regularity of Lebesgue measure, or approximation by finite unions of intervals. Translation invariance and uniform continuity give
where the approximation error is first made arbitrarily small. Equivalently, .
Choose a return time so small modulo one that for each ,
This is possible because multiplication by each fixed is continuous on the circle group and translation continuity in L1 on the circle applies to the finitely many translates. The union bound now gives
This proves the Furstenberg multiple recurrence theorem for every circle rotation with its normalized Lebesgue measure, including both rational and irrational angles.
Fix the normalization
The Lasso penalty uses the L1 norm. All arguments below apply to every minimizer; uniqueness of the coefficient vector is not assumed.
The Karush-Kuhn-Tucker conditions are the subgradient optimality condition
These conditions are necessary and sufficient because the objective is a convex function. They use the subgradient of the absolute value, coordinate by coordinate.
Put . Centered columns imply , so the subtraction of in the model contributes nothing to the score. The Karush-Kuhn-Tucker conditions become
Multiply by . Since and , this gives the prediction inequality from Lasso stationarity:
This is the desired version of the Basic inequality for the Lasso. Using stationarity directly preserves the coefficient on the squared prediction norm.
For the Gaussian score event for the Lasso, write for column . The assumed Euclidean norm and the multivariate normal distribution of give
The may be correlated, which does not affect the union bound. Use the following sharp two-sided Gaussian tail bound, valid for with the standard normal distribution and every :
For completeness, the difference satisfies , , and as . It first increases and then decreases to zero, so it is nonnegative. This bound avoids introducing an unnecessary factor two.
Here is the standard normal distribution function. With , the union bound yields
Consequently
The score event uses the supremum norm of ; the centering term still vanishes because .
We next work deterministically on . Let
By Holder inequality, . Since , the triangle inequality gives
Substituting both inequalities into the Basic inequality for the Lasso produces
Because , this also proves the relevant Lasso cone condition, . The assumed Compatibility condition for the Lasso therefore applies to :
Set . Combining these inequalities gives
If , discarding the nonnegative term shows . If , compatibility gives , and the preceding inequality then gives . In either case , so
This is a fast prediction bound and an inactive-coordinate error bound on the same event. It is the fast-rate Lasso prediction bound under compatibility.
Finally, the same argument bounds the active-coordinate L1 norm by
If , then , and . When this implies ; when it implies . Thus the sign function obeys
This is Lasso sign recovery under compatibility. It concerns the sufficiently large nonzero coefficients; it does not assert that every estimated inactive coefficient is zero.
Write for a positive-definite matrix. Up to a constant independent of the parameters, the log-likelihood of the multivariate normal distribution is
The identity
shows that, for each , the unique optimizing mean is . Using the matrix trace to rewrite the first quadratic sum, the profile likelihood is
Therefore maximum likelihood estimation of the precision matrix, whenever the maximum exists, is exactly
This is the profile likelihood for a Gaussian precision matrix. The sample covariance matrix here uses divisor , as appropriate for maximum likelihood estimation.
There is an important rank qualification in the printed assumptions. If , differentiation gives , so the unique minimizer is . Uniqueness follows because the negative log-determinant is a strictly convex function on the positive-definite matrices. But full column rank of the uncentered alone does not imply : the rank of a centered sample covariance matrix depends on the centered design. For example, with and raw data matrix ,
Although has full column rank, setting gives objective . Thus an unpenalized maximizer need not exist under the literal raw-rank assumption. The likelihood reduction remains valid; existence requires full column rank of the centered data, equivalently . For the nonsingular Gaussian model this holds almost surely when .
Use the entrywise-penalty convention for the Graphical Lasso:
The sum is the entrywise matrix L1 norm. Its Graphical-Lasso Karush-Kuhn-Tucker conditions are
Here is symmetric; its nonzero-entry values are , and its zero-entry values are any numbers in . These are the subgradient optimality condition for this convex optimization problem. Since a positive-definite matrix has positive diagonal, and . Counting both off-diagonal entries in the penalty and in the trace makes the same appear in their stationarity equations. The common variant penalizing only off-diagonal entries instead sets ; the off-diagonal equations and the column argument below are unchanged.
For the given column problem, abbreviate
Its smooth part expands to plus a constant, because is symmetric and . The matrix is a positive-definite matrix, so this quadratic is a strictly convex function. Adding the convex L1 norm penalty preserves strict convexity, and the objective tends to infinity as . The minimizer exists and is unique. Its Karush-Kuhn-Tucker conditions are
We identify that minimizer directly from the precision matrix. Write and define
The denominator is positive. The off-diagonal part of column of gives
On the other hand, the off-diagonal Graphical-Lasso Karush-Kuhn-Tucker conditions read
Because , the coordinates of have the opposite signs from those of and vanish at exactly the same positions. The subgradient of the absolute value therefore satisfies
It follows that
These are exactly the Lasso column problem's Karush-Kuhn-Tucker conditions. Its unique minimizer must be , yielding
This is the Graphical Lasso column regression identity. The minus sign is essential: it converts the precision-entry subgradient into the regression-coefficient subgradient. The argument uses the inverse identity directly, so it does not require proving the supplied block inverse formula.
The interior of the diamond is . Its Minkowski functional, or the uniqueness argument already proved, gives the L1 norm:
The final unheaded set cannot be an open unit ball of a norm, because every norm ball is convex by homogeneity and the triangle inequality. Both and lie in : their squared-distance expression is . Their midpoint has that expression
so . Thus . The strict inequalities in the definition are respected by this explicit counterexample.
The Martingale convergence theorem in its -bounded form says that a martingale with has an integrable limit and
The norm bound on the limit follows from Fatou lemma. Boundedness in by itself does not imply convergence in L1. The uniformly integrable martingale convergence theorem gives the stronger conclusion: if is uniformly integrable, then both almost surely and in L1 norm, and . Conversely, convergence in L1 implies uniform integrability.
For the distinction, the fair-coin doubling martingale has expectation one for every but converges almost surely to zero. Its L1 norm remains one, so its convergence is not in L1 norm. These two formulations specify exactly which hypothesis is needed in part (d).
Work on as a probability space, with Lebesgue measure of total mass one. Let be the filtration generated by the level- half-open dyadic cells together with the separate null cell . Define the dyadic slope martingale by
The endpoint may be assigned any value, since it is a null set for Lebesgue measure. The two child slopes average to their parent slope, by telescoping the two increments. Therefore almost everywhere, so this is a martingale. The Lipschitz condition gives everywhere except possibly at the freely chosen endpoint, where we take zero.
Apply the Lp martingale convergence theorem with . Its limit has almost everywhere and in L1 norm as well, by Cauchy-Schwarz inequality. Choose a measurable representative of and set it to zero on any exceptional null set; it is then a bounded measurable function on the entire interval.
Set . Telescoping at the grid points shows that is the linear interpolation of on the dyadic grid. The Lipschitz condition gives . Also
The two uniform limits coincide, giving the absolutely continuous function representation
The chosen bound holds for every after the null-set modification; the integral identity holds for every simultaneously.
Use the same dyadic slope martingale and dyadic filtration as in (c). Each is integrable, since it takes finitely many finite values. On each dyadic cell, the absolute slope is times the absolute endpoint increment. Consequently the hypothesis in the PDF is exactly
Thus is uniformly integrable. It is also bounded in L1 norm: choose a finite at which the supremum of the tails is finite, and use . The uniformly integrable martingale convergence theorem supplies with in L1 norm.
The functions are again the dyadic linear interpolations of . Since is continuous on a compact interval, it is uniformly continuous, and , where is its modulus of continuity. On the other hand, the integral of is uniformly bounded in absolute value by . Hence the dyadic slope-tail criterion for absolute continuity gives
No boundedness of is asserted here; the tail condition permits integrable densities that are unbounded.
Put , the uniform bound on the supremum norm of the gradient. The fundamental theorem of calculus on a line segment, together with the inner product inequality , gives
Thus is Lipschitz continuous with respect to the L1 norm. The Gaussian random variables comprising and have finite exponential moments of their absolute values. More explicitly, for , Hölder's inequality gives . Consequently and every exponential expression below are finite, including when the covariance matrix is singular.
Use the convex function . Since is an independent copy of , Jensen inequality for the conditional expectation gives
For , define the Gaussian rotation of independent copies
Writing , both rotated covariance matrices are and their cross-covariance is zero. The pair has a centered multivariate normal distribution, so its components are independent and has the same probability law as . This calculation uses no inverse of and therefore also proves the assertion for degenerate multivariate normal distributions.
The path runs from to . The chain rule and the fundamental theorem of calculus give the pathwise identity
Apply Jensen inequality to the uniform probability measure on . Then use Tonelli theorem and the Gaussian rotation of independent copies to obtain
This proves the Gaussian rotation interpolation inequality in the required form:
The stated almost-sure domination now gives an exponential moment of an absolute standard normal variable bound. If , completing the square in the standard normal density yields
where is the standard normal distribution function. Hence Markov inequality gives, for every ,
The exponent is minimized at . Therefore the resulting Gaussian tail bound is
No independence of from or is needed: only the given almost-sure domination is used.
For multi-indices , define
These seminorms define the Fréchet space topology of the Schwartz space: exactly when every tends to zero. An equivalent increasing family is
A tempered distribution is a continuous linear functional on this space. Equivalently, for some . The usual weak convergence of tempered distributions means for every fixed Schwartz function. The strong dual topology instead requires uniform convergence on every subset of the Schwartz space that is a bounded set in a topological vector space; the Fourier maps below are continuous in both topologies.
Using , differentiation under the integral and integration by parts give
The omitted coefficient has modulus one. By the Leibniz rule, every integrand is a finite sum of a polynomial times a derivative of . Inserting the integrable weight bounds its L1 norm by finitely many Schwartz space seminorms. In particular . Thus the Fourier transform maps continuously into itself.
For completeness, the Fourier inversion theorem follows here by Gaussian regularization. The inverse transform of is . The Fourier transform of a Gaussian and Fubini's theorem show
As , the right side tends to by the approximate identity property, while the left side tends to the undamped inverse integral by dominated convergence theorem, since . Hence
The inverse is times reflection composed with the continuous Fourier transform, and is therefore continuous on the Schwartz space. This proves a continuous linear isomorphism with continuous inverse.
Define the Fourier transform of a tempered distribution by
The continuous map on Schwartz space makes this a tempered distribution; the transpose of supplies its inverse. Pointwise convergence of pairings proves weak continuity. For the strong dual topology, the transform of a bounded set of Schwartz functions is bounded, so uniform convergence of pairings on bounded sets proves continuity of both Fourier maps there too.
Now let be a rotation matrix in the special orthogonal group and write . A change of variables with unit Jacobian gives
The rotation equivariance of the Fourier transform on tempered distributions follows by duality:
Therefore . Applying this identity and the inverse Fourier transform gives the two directions:
This is precisely preservation of radial tempered distributions. For , acts transitively on spheres, so an invariant smooth function is an ordinary radial function.
Use localization and cancellation of the assumed interval-adapted wavelet basis: at scale , every detail wavelet has support diameter at most , L1 norm at most , and annihilates polynomials of degree less than . These bounds also apply to the modified boundary wavelets; a restriction of an arbitrary whole-line wavelet without the cancellation-preserving boundary construction would not suffice. The finitely many coarse scaling functions can be treated separately.
Write with as above. Because , we have . Choose in the support of a function of , and let be the degree- Taylor polynomial of . The Hölder-Taylor remainder bound gives
For this is the Hölder seminorm bound. For , the integral Taylor remainder is bounded using , whose magnitude is at most .
The vanishing moments remove from the inner product. Applying the remainder estimate and the L1 norm bound on the localized support yields the wavelet coefficient decay for Hölder functions
Thus
Here depends on the fixed wavelet family, the exponent and boundary construction, but not on . The proof uses the assumed regularity and moments; it does not identify the minimal Daubechies wavelet order with its differentiability order, which need not coincide.
It is equivalent to minimize or its th power , because taking the th root is strictly increasing. For , this is an Lq quasi-norm, while gives the L1 norm.
Assume the Lq null space property of order . Let have support of a vector with , and let be any distinct feasible vector. Then . The triangle inequality for complex magnitudes and the subadditivity give
For the same inequality follows directly from the triangle inequality. Since off ,
Thus is the unique minimizer.
Conversely, suppose every -sparse vector is the unique minimizer for its measurements. Fix and any with . The vectors and are distinct and satisfy , since . Uniqueness for this therefore implies , exactly the required strict inequality. Hence
The quantifier is uniform over the entire sparse class. Recovery of one particular signed vector alone would not imply this null space property.
For real , its total variation seminorm on a domain is
The vector fields have compact support inside , so no boundary term is charged. Taking both and makes this equivalent to a supremum of absolute pairings. When finite, the distributional derivative is a finite vector-valued Radon measure, and the displayed supremum is its total variation norm of a measure. For a smooth function it equals .
The function of bounded variation on a domain belongs to the Banach space
The L1 norm is needed because variation alone vanishes on constant functions and is only a seminorm. No smoothness of is part of this definition.
The L1 norm properties of absolute homogeneity and the triangle inequality follow from the corresponding properties of the absolute value and the integral. If , the nonnegative continuous function has zero integral. Were , continuity would make it bounded below by a positive number on a nontrivial interval, contradicting the zero integral. Thus , proving positive definiteness.
Put and choose the continuous cutoff function
Then , , and is supported in the two endpoint intervals of total length . Therefore
Now suppose for every . Apply this to the above . By the uniform norm bound,
Hence , and continuity again gives .
A metric space metric satisfies positivity, exactly when , symmetry, and the triangle inequality. A subset is an open set when every has some with the open ball .
Both and are open. An arbitrary union of open sets is open because a point lies in one member supplying a ball. A finite intersection is open because the minimum of the finitely many available radii supplies a ball. Thus the metric-open sets satisfy the topology axioms.
For , the Cauchy-Schwarz inequality gives , so the topology is at least as fine as the topology. To see strictness, define the continuous triangular spikes
Then
Thus converges to zero in the L1 norm but not in the L2 norm. The two metrics induce different topologies.