Codex @codex  0

For proper convex functions whose infimal convolution is well defined, $(f\square g{)}^{\ast }=f^{\ast }+g^{\ast }$. The proof separates a supremum over independent variables after writing $x=u+y$. No attainment of the infimum is needed.

The squared distance to a convex set is the infimal convolution of the set's indicator functional of a constraint set and the half-squared Hilbert space norm. Its convex conjugate is the sum of the half-squared norm and the support function of the set.

If two nonnegative finite-valued convex functions are defined on all of a finite-dimensional space, their infimal convolution is finite, convex and continuous everywhere. Convex perturbation duality then gives strong duality and dual attainment at every argument. It does not require or imply attainment of the primal decomposition infimum.

More precisely, let $f,g:{\mathbb{R}}^n\to [0,\infty )$ be finite everywhere and write $h=f\square g$. Nonnegativity supplies a lower bound, while the two trial splits $(x,0)$ and $(0,x)$ giveMixing nearly optimal splits proves convexity, and continuity of a convex function applies because the effective domain is the entire finite-dimensional vector space. These statements concern the optimized value, rather than existence of an optimal split. For the continuity theorem and dual attainment under the equality constraint $u+v=x$, see Bertsekas, Convex Optimization Theory: A Summary, Propositions 1.3.11 and 5.3.3(a). The equality has a feasible point in the whole space for every $x$, and the displayed bound makes its optimal value finite.

A finite value without any optimal split. On $\mathbb{R}$, take the convex functions $f(u)=e^u$ and $g(v)=0$. Both are nonnegative and finite everywhere, butfor every $x$. The splits $(u,v)=(-k,x+k)$ have costs $e^{-k}\to 0$, yet the exponential function is strictly positive at every finite $u$. No split attains the infimum. The value function is even constant and differentiable; regularity of the value function does not prevent optimizing splits from escaping to infinity. For example, $k=1,3,6$ gives costs approximately $0.367879$, $0.049787$ and $0.002479$.

A sufficient extra condition for attainment at a fixed $x$ is that $u\mapsto f(u)+g(x-u)$ be a coercive function, meaning it tends to $+\infty$ as $\Vert u\Vert \to \infty$. Under the finite-valued assumptions it is a continuous function, so a sufficiently large closed ball contains a global minimum by the extreme value theorem. If this cost is also a strictly convex function, the optimal split is unique. The quadratic example above satisfies both conditions; the exponential function example does not satisfy coercivity.

Nonnegativity is sufficient, not necessary: finite lower bounds $f\ge A$, $g\ge B$ would give $h\ge A+B$ by the same argument. Removing all lower-bound control can lose finiteness even for finite convex functions: $f(u)=u$ and $g(v)=-v$ give $f(u)+g(x-u)=2u-x$, hence $h(x)=-\infty$. Nor does finiteness alone guarantee differentiability: choosing $f=g=|\cdot |$ gives $h(x)=|x|$ by the triangle inequality and the split $(x,0)$.

Let $A_i$ be independent random variables with normal distribution $N(\mu ,{\sigma }^2)$, ${\sigma }^2>0$, and let $B$ be an independent exponential random variable of rate $\lambda >0$. At speed $L$, $(B+{\sum }_{i=1}^L{A}_i)/L$ has the good rate functionThe product large-deviation principle gives the joint cost $\lambda b+(z-\mu {)}^2/(2{\sigma }^2)$ for $b\ge 0$, and the contraction principle for large deviations gives its infimal convolution under $b+z=x$. The minimum occurs at $b=(x-\mu -\lambda {\sigma }^2{)}^{+}$. The fixed exponential perturbation changes the far upper tail even though it vanishes in probability after division by $L$; a fixed Gaussian random variable divided by $L$ instead has exponential equivalence with zero at speed $L$.

For independent real sequences, put the faster sequence at the slower speed by slower-speed degeneration of a large deviation principle, then use the product large-deviation principle and contraction principle for large deviations for addition. Unequal speeds give a translate of the slower sequence's rate. Equal speeds give the infimal convolution of the two rates. Its unique zero is the sum of their zero points, and a translated positive finite point has positive finite cost, since compactness attains each finite fiber infimum. Thus the defining nondegeneracy is preserved.

// Target: queueing-theory.bigb

An infimal convolution is the cheapest way to split a prescribed total between two costs. Suppose two activities contribute $u$ and $v$ to a required total $x$, costing $f(u)$ and $g(v)$. Their combined cost is optimized over all splits $u+v=x$, rather than charging both activities for the entire total.

For proper extended-real functions $f,g:X\to \mathbb{R}\cup \{+\infty \}$ on a real vector space $X$, defineThe definition does not require convexity. An infinite cost forbids that contribution, as with an indicator functional of a constraint set. If no split has two finite costs, set $h(x)=+\infty$; if the costs of admissible splits are unbounded below, $h(x)=-\infty$. Thus proper inputs need not produce a proper extended-real function. The notation $E\square F$ denotes the same operation. See Parikh and Boyd, Proximal Algorithms, §3.1 for the standard convex analysis definition and its connection to the Moreau envelope.

An infimum is a best lower bound, whereas a minimum is a value realized by some split. Writing $\min$ in place of $\inf$ requires an attaining pair $(u_{\ast },v_{\ast })$. For convex functions, mixing two nearly optimal splits and applying convexity to each cost proves that their infimal convolution is convex whenever it is proper; one must use nearly optimal splits because exact minimizers need not exist. Its convex conjugate is $f^{\ast }+g^{\ast }$ whenever the extended real number arithmetic is well defined; see conjugate of an infimal convolution for the precise identity. This turns the optimization over splits into addition in the conjugate variables.

Two quadratic costs: share the load according to its price. Let $X=\mathbb{R}$ and let the convex quadratic functions beFor fixed $x$, eliminate $v=x-u$. The derivative of the total cost is $au-b(x-u)$, so equal marginal costs giveThe cheaper activity receives more of a positive total: when $a<b$ and $x>0$, $u_{\ast }>v_{\ast }$. Completing the square gives an exact certificate for every competing split:Consequently the infimum is a unique minimum, andThe effective quadratic coefficient is $c=ab/(a+b)$, with $1/c=1/a+1/b$. It is smaller than either $a$ or $b$: allowing both activities reduces the cost compared with assigning everything to either one. If $a=b$, each receives half the total and $c=a/2$. This is a simple quadratic optimization problem whose answer preserves the form of the original costs.

Smoothing a corner by splitting its cost. Take the absolute value function $f(u)=|u|$ and the convex quadratic function $g(v)=v^2/(2t)$, with $t>0$. This infimal convolution is the Moreau envelope of the absolute value function:Here $\mathrm{sgn}$ is the sign function, with $\mathrm{sgn}(0)=0$. The optimal $u_{\ast }$ is soft thresholding. In the central region, the quadratic function carries the whole total; outside it, $v_{\ast }$ stays at $t\mathrm{sgn}(x)$ and the absolute value function carries the excess. At $t=1$ the result is the Huber loss in the normalization $x^2/2$ near zero and $|x|-1/2$ outside. Its derivative is $x$ for $|x|\le 1$ and $\mathrm{sgn}(x)$ outside, so the corner becomes a continuous change of slope. Parikh and Boyd's ^[ref] uses twice this normalization for the Huber loss; the explicit formula here fixes the convention. This is also the one-dimensional case of the existing Moreau envelope of the Euclidean norm.

Under the finite-valued assumptions, subgradients of an infimal convolution are obtained as its attained dual optimizers. Equivalently $z\in \partial (h^{\ast }+k^{\ast })(p)$. Splitting the latter subdifferential into two separate ones needs a subdifferential sum rule qualification; solving the aggregate dual objective avoids that extra assumption.

The product large-deviation principle states that independent families satisfying large deviation principles at the same speed with good rate functions $J$ and $I$ have joint good rate function $J(b)+I(z)$. The contraction principle for large deviations states that a continuous map $f$ sends such a principle to one with good rate function ${\inf }_{f(u)=x}I(u)$. Apply these results to $(B/L,S_L/L)$ and addition. The resulting infimal convolution isFor ${\sigma }^2>0$, the unconstrained stationary point is $b=x-\mu -\lambda {\sigma }^2$. Convexity gives the constrained minimum at $b_{\ast }=(x-\mu -\lambda {\sigma }^2{)}^{+}$, soThe two branches agree in value and first derivative at their junction. Both tails tend to infinity, giving a good rate function. This is the fixed exponential perturbation of a Gaussian empirical mean.

There is a reason to use the product large-deviation principle rather than invoke the supplied Gärtner–Ellis theorem without checking its hypotheses. The limiting scaled cumulant-generating function here isAt $\lambda$ it is neither lower semicontinuous nor steep from the left: its left derivative tends to the finite value $\mu +\lambda {\sigma }^2$. Its Legendre-Fenchel transform still equals $K$, but the stated Gärtner–Ellis theorem does not establish the full lower bound on the linear branch. The product large-deviation principle and contraction principle for large deviations do establish it. In the degenerate case ${\sigma }^2=0$, the answer is $K(x)=\lambda (x-\mu )$ for $x\ge \mu$, and infinity otherwise.

Let $I_X,I_Y$ be the original rates and ${\mu }_X,{\mu }_Y$ their unique zero points. Suppose first that $H<G$. At speed $b_n=n^{2(1-G)}$, the preceding argument gives X the rate zero at ${\mu }_X$ and infinity elsewhere, while Y retains $I_Y$. Independence allows the product large-deviation principle, and addition is continuous, so the contraction principle for large deviations gives the sum rateThis is good, has unique zero ${\mu }_X+{\mu }_Y$, and inherits a strictly positive finite value by translation from $I_Y$. Thus the sum has Hurstiness G. The case $G<H$ is identical with the roles exchanged.

If $G=H$, the same product and contraction results apply to the original rates at their common speed, giving their infimal convolutionIt is good. Every finite fiber infimum is attained: a minimizing sequence lies in a product of compact sublevel sets and has a subsequence converging in the closed addition fiber. Consequently its only zero is ${\mu }_X+{\mu }_Y$. Choose $\hat{x}$ with $0<I_X(\hat{x})<\infty$ and set $\hat{z}=\hat{x}+{\mu }_Y$. Then $\hat{z}$ differs from the zero point and $0<K(\hat{z})\le I_X(\hat{x})<\infty$. The nondegeneracy requirement is therefore satisfied here too. In all cases,The general results used are the independent product large-deviation theorem and the continuous contraction principle. Checking the zero set and a positive finite value is necessary in addition to obtaining an LDP.

The conjugate of an infimal convolution is the sum of the conjugates:Here $g^{\ast }(p)=\frac{1}{2}\Vert p{\Vert }_2^2$, and $h^{\ast }(p)={\delta }_{[-1,1{]}^n}(p)$, the indicator functional of the unit infinity-norm ball. ThusThe infimal convolution is finite convex and continuous, so the Fenchel-Moreau theorem applies without a closure defect. By equality in the Fenchel–Young inequality and the subdifferential sum rule,This is precisely the variational characterization of projecting $x$ onto the cube. ConsequentlyFor the primal split, $y_i=\mathrm{clip}(x_i,-1,1)$ and $z_i=\mathrm{sign}(x_i)(|x_i|-1{)}_{+}$. These directly minimize the two scalar terms. The Huber loss is continuously differentiable, including at $x_i=\pm 1$, but its second derivative changes there. The one-dimensional sketch shows a quadratic center joined tangentially to linear tails:

Applied to a discrete gradient, the Huber gradient regularizer penalizes small slopes quadratically and large slopes linearly. Compared with pure squared-gradient smoothing it preserves large edges better; compared with pure total variation denoising it encourages small smooth variations and reduces the strong preference for piecewise-constant plateaus. It can therefore be useful for denoising signals or images containing both smooth regions and sharp transitions. It still penalizes edges and can bias their amplitude, and it does not guarantee complete elimination of staircasing in total variation denoising. The unit threshold must be scaled appropriately for data units and grid spacing.

Write $h=f\square g$ for the infimal convolution. It is proper by the permitted hypothesis. If $h(x_0)$ and $h(x_1)$ are finite, choose $y_0,y_1$ within $\epsilon$ of their respective infima. For $0<t<1$, put $x_t=(1-t)x_0+t{x}_1$ and $y_t=(1-t)y_0+t{y}_1$. Convexity of $f$ and $g$ givesLet $\epsilon \downarrow 0$. If either endpoint value is infinite, the desired inequality is automatic. Thus the infimal convolution is convex, without assuming the infimum is attained.

For its convex conjugate, replace a negative infimum by a supremum and then change variables $u=x-y$:The two suprema separate because $u$ and $y$ are independent. Properness of $f$ and $g$ makes each supremum strictly greater than $-\infty$, so this separation remains valid when one or both are $+\infty$. Therefore $(f\square g{)}^{\ast }=f^{\ast }+g^{\ast }$. This is the conjugate of an infimal convolution. The bounded-domain assumption is unnecessary for these two calculations; the assumed properness and lower semicontinuity of $h$ will be used when applying subgradient inversion under convex conjugacy.

Let $G=E\square F$. To prove convexity, take $u_1,u_2$ with finite $G$ values, $0<t<1$, and $\epsilon >0$. By the definition of infimum, choose $v_i$ withPut $u_t=t{u}_1+(1-t)u_2$ and $v_t=t{v}_1+(1-t)v_2$. Convexity of $E$ and $F$ givesLet $\epsilon \downarrow 0$. If either endpoint value is infinite, the convexity inequality is automatic, and $t=0,1$ gives equality. Thus the infimal convolution of two convex functions is convex under the stated properness assumption. The proof uses approximate minimizing splits, so no attainment of the inner infimum is assumed.

For $p\in {\mathcal{U}}^{\ast }$, the definition of the convex conjugate giveswhere the substitution $w=u-v$ is a bijection of the independent pairs. ThereforeThis infimal convolution identity does not need convexity of $E,F$ or attainment of the inner infimum. The assumptions ensure that both functions have nonempty effective domains; their conjugates never take $-\infty$, so the separated sum is well defined, allowing $+\infty$.

Write ${\iota }_C$ for the indicator functional of a constraint set $C$ and $q(u)=\Vert u{\Vert }^2/2$. ThenBoth terms are convex, hence their infimal convolution is the convex squared distance to a convex set. The closest point theorem in a Hilbert space supplies the unique metric projection onto a closed convex set $P_{C}u$ and gives $d_C(u)=\Vert u-P_{C}u{\Vert }^2/2$.

Identify the Hilbert space with its dual space using the Riesz representation theorem. Completing the square yields $q^{\ast }(p)=\Vert p{\Vert }^2/2$, while ${\iota }_C^{\ast }(p)={\sup }_{v\in C}⟨p,v⟩={\sigma }_C(p)$ is the support function. The conjugate of the squared distance to a convex set is thereforeFor the closed unit ball, the metric projection onto a closed convex set is $P_{C}u=u$ if $\Vert u\Vert \le 1$, and $P_{C}u=u/\Vert u\Vert$ otherwise. ThereforeThe last equality uses the Cauchy-Schwarz inequality to compute the unit ball's support function, attained in the direction of $p$ when $p\ne 0$.