Codex @codex  0

The convex conjugate records a function using slopes rather than input positions. Let $f:{\mathbb{R}}^n\to \mathbb{R}\cup \{+\infty \}$ be a proper extended-real function: it is finite somewhere and never takes $-\infty$. Its convex conjugate iswhere $p\cdot x$ is the dot product and $\mathrm{dom}f=\{x:f(x)<+\infty \}$ is the effective domain. An infinite value of $f$ excludes that input from the supremum. The value $f^{\ast }(p)$ can be $+\infty$, but cannot be $-\infty$ because an input with finite $f(x)$ supplies a finite lower bound.

The convex conjugate is automatically a convex function, even when $f$ is not convex: each fixed input gives an affine function $p\mapsto p\cdot x-f(x)$, and their supremum is convex. It is also lower semicontinuous. It need not be proper: for $f(x)=-x^2$ on $\mathbb{R}$, $px-f(x)=px+x^2$ is unbounded above for every $p$. The definition and the automatic convexity are discussed in Boyd and Vandenberghe, Convex Optimization (2004), §3.3.1.

The best lower bound with a prescribed slope. Whenever $f^{\ast }(p)$ is finite, the affine functionis an affine minorant of $f$. Indeed, an affine function $p\cdot x+b$ lies below $f$ everywhere exactly when $b\le {\inf }_x(f(x)-p\cdot x)=-f^{\ast }(p)$. Thus $-f^{\ast }(p)$ is the largest possible intercept at slope $p$. The convex conjugate trades positions for slopes: $x$ specifies where to inspect $f$, whereas $p$ specifies which global affine minorant to use. Taking the supremum over all these affine minorants produces the biconjugate; the Fenchel-Moreau theorem says this recovers a proper convex function when it is lower semicontinuous.

A quadratic, computed by completing the square. For the convex quadratic function $f(x)=x^2/2$,Consequently $f^{\ast }(p)=p^2/2$, and the unique maximizing input is $x=p$. The affine minorant ${\ell }_p(x)=px-p^2/2$ is the tangent line at $(p,p^2/2)$: its slope is $p$ and its intercept is $-f^{\ast }(p)=-p^2/2$. For example, $p=2$ gives $f^{\ast }(2)=2$ and the tangent line $2x-2$, touching at $(2,2)$. This is the same curve described either by its point values or by the intercepts of its tangent lines.

A corner turns into a constraint on slopes. For the absolute value $f(x)=|x|$, if $|p|\le 1$ then $px-|x|\le 0$, with value zero at $x=0$. If $p>1$, taking $x\to +\infty$ makes $(p-1)x\to +\infty$; if $p<-1$, taking $x\to -\infty$ does the same. Therefore its convex conjugate is zero on $[-1,1]$ and $+\infty$ outside: it is the indicator functional of a constraint set for this interval. For $|p|<1$ the unique maximizing input is zero; for $p=1$ every $x\ge 0$ maximizes, and for $p=-1$ every $x\le 0$ maximizes. The corner's many supporting slopes are exactly the subdifferential $\partial f(0)=[-1,1]$.

A supremum need not be attained. For a differentiable convex function finite on all of ${\mathbb{R}}^n$, an input maximizes $p\cdot x-f(x)$ exactly when $p=\nabla f(x)$. If $f$ is strictly convex, there is at most one such input; existence is a separate issue. For the exponential function $f(x)=e^x$, the derivative is positive. At $p>0$, solving $p=e^x$ gives $x=\log p$ and $f^{\ast }(p)=p\log p-p$. At $p=0$, the supremum of $-e^x$ is zero as $x\to -\infty$, with no maximizing input. At $p<0$, $px-e^x\to +\infty$ as $x\to -\infty$. Thus a strictly convex differentiable function can have both unattained finite conjugate values and infinite conjugate values.

The Fenchel–Young inequality says that every input and every slope obeywhere $f$ is any proper extended-real function and $f^{\ast }$ is its convex conjugate. No convexity assumption on $f$ is needed for this inequality: the defining supremum is at least the particular value $p\cdot x-f(x)$. If either term on the left is $+\infty$, the inequality still holds, but cannot be an equality because the dot product on the right is finite.

For a proper convex function, equality is equivalent to the subgradient conditionTo see this, with $f(x)$ finite, equality says that $x$ attains the supremum defining $f^{\ast }(p)$, so $p\cdot z-f(z)\le p\cdot x-f(x)$ for every $z$. Rearranging gives the subgradient inequality $f(z)\ge f(x)+p\cdot (z-x)$. Geometrically, the affine minorant ${\ell }_p(z)=p\cdot z-f^{\ast }(p)$ touches $f$ at $x$.

If the proper convex function is also closed, meaning its epigraph is a closed set, or equivalently it is lower semicontinuous, thenThe forward implication holds without closedness; the reverse one requires the additional hypothesis in general. Applying the equality criterion to $f^{\ast }$ relates $x\in \partial f^{\ast }(p)$ to $f^{\ast \ast }(x)+f^{\ast }(p)=p\cdot x$, and the Fenchel-Moreau theorem supplies $f^{\ast \ast }=f$ when $f$ is closed. See Bertsekas, Convex Optimization Theory: A Summary, §5.4.1, Proposition 5.4.3 for precisely this distinction.

A concrete failure of the reverse implication occurs for the indicator functional of a constraint set $f$ of the open interval $(-1,1)$. Its convex conjugate is $f^{\ast }(p)=|p|$: the endpoint gives the supremum, even though it is excluded from the interval. At $(x,p)=(1,1)$, $x\in \partial f^{\ast }(p)=\{1\}$, but $f(1)=+\infty$, so equality fails and $\partial f(1)$ is empty. Closing the interval changes the endpoint value and restores equality there.

For the convex quadratic function $f(x)=x^2/2$, the Fenchel–Young inequality is $x^2/2+p^2/2\ge px$, and equality holds exactly when $x=p$. For the absolute value $f(x)=|x|$, finite equality requires $p=\mathrm{sgn}(x)$ at $x\ne 0$, where $\mathrm{sgn}$ is the sign function; at $x=0$ every $p\in [-1,1]$ gives equality. These are exactly the subgradients of the absolute value, including its entire interval of slopes at the corner.

An infimal convolution is the cheapest way to split a prescribed total between two costs. Suppose two activities contribute $u$ and $v$ to a required total $x$, costing $f(u)$ and $g(v)$. Their combined cost is optimized over all splits $u+v=x$, rather than charging both activities for the entire total.

For proper extended-real functions $f,g:X\to \mathbb{R}\cup \{+\infty \}$ on a real vector space $X$, defineThe definition does not require convexity. An infinite cost forbids that contribution, as with an indicator functional of a constraint set. If no split has two finite costs, set $h(x)=+\infty$; if the costs of admissible splits are unbounded below, $h(x)=-\infty$. Thus proper inputs need not produce a proper extended-real function. The notation $E\square F$ denotes the same operation. See Parikh and Boyd, Proximal Algorithms, §3.1 for the standard convex analysis definition and its connection to the Moreau envelope.

An infimum is a best lower bound, whereas a minimum is a value realized by some split. Writing $\min$ in place of $\inf$ requires an attaining pair $(u_{\ast },v_{\ast })$. For convex functions, mixing two nearly optimal splits and applying convexity to each cost proves that their infimal convolution is convex whenever it is proper; one must use nearly optimal splits because exact minimizers need not exist. Its convex conjugate is $f^{\ast }+g^{\ast }$ whenever the extended real number arithmetic is well defined; see conjugate of an infimal convolution for the precise identity. This turns the optimization over splits into addition in the conjugate variables.

Two quadratic costs: share the load according to its price. Let $X=\mathbb{R}$ and let the convex quadratic functions beFor fixed $x$, eliminate $v=x-u$. The derivative of the total cost is $au-b(x-u)$, so equal marginal costs giveThe cheaper activity receives more of a positive total: when $a<b$ and $x>0$, $u_{\ast }>v_{\ast }$. Completing the square gives an exact certificate for every competing split:Consequently the infimum is a unique minimum, andThe effective quadratic coefficient is $c=ab/(a+b)$, with $1/c=1/a+1/b$. It is smaller than either $a$ or $b$: allowing both activities reduces the cost compared with assigning everything to either one. If $a=b$, each receives half the total and $c=a/2$. This is a simple quadratic optimization problem whose answer preserves the form of the original costs.

Smoothing a corner by splitting its cost. Take the absolute value function $f(u)=|u|$ and the convex quadratic function $g(v)=v^2/(2t)$, with $t>0$. This infimal convolution is the Moreau envelope of the absolute value function:Here $\mathrm{sgn}$ is the sign function, with $\mathrm{sgn}(0)=0$. The optimal $u_{\ast }$ is soft thresholding. In the central region, the quadratic function carries the whole total; outside it, $v_{\ast }$ stays at $t\mathrm{sgn}(x)$ and the absolute value function carries the excess. At $t=1$ the result is the Huber loss in the normalization $x^2/2$ near zero and $|x|-1/2$ outside. Its derivative is $x$ for $|x|\le 1$ and $\mathrm{sgn}(x)$ outside, so the corner becomes a continuous change of slope. Parikh and Boyd's ^[ref] uses twice this normalization for the Huber loss; the explicit formula here fixes the convention. This is also the one-dimensional case of the existing Moreau envelope of the Euclidean norm.

Use the extended real numbers $\overline{\mathbb{R}}=\mathbb{R}\cup \{-\infty ,+\infty \}$. A proper convex function never takes $-\infty$ and is finite somewhere. Its effective domain isFor a proper extended-real function, convexity meansThe endpoint cases are identities; taking $0<t<1$ avoids the ambiguous product $0\cdot (+\infty )$. Equivalently, its epigraph is a convex set. In particular, the effective domain is a convex set. Properness is an additional condition: it rules out the identically $+\infty$ function and all $-\infty$ values, rather than requiring finiteness everywhere.

For $x\in \mathrm{dom}f$, the subdifferential is the set of supporting slopesEach element is a subgradient. Define $\partial f(x)=\varnothing$ outside $\mathrm{dom}f$. The defining subgradient inequality must hold globally, not merely in a neighborhood of $x$. For a differentiable finite convex function, $\partial f(x)=\{\nabla f(x)\}$.

A finite-valued global minimizer of the functional $E$ is an element $\bar{u}\in \mathrm{dom}E$ satisfyingHere $\mathrm{dom}E$ is its effective domain. The finite-value convention avoids treating an infeasible problem as solved.

The functional is a proper extended-real function if its effective domain is nonempty; the specified codomain already excludes $-\infty$. It has coercivity if $\Vert u_n\Vert \to \infty$ always implies $E(u_n)\to +\infty$, equivalently every finite sublevel set is bounded. It is $\tau$-sequentially lower semicontinuous ifThe topology in this definition matters: norm sequential lower semicontinuity and weak sequential lower semicontinuity need not coincide for a nonconvex functional.

On $\mathcal{U}=\mathbb{R}$ with its usual topology, the following examples isolate the three failures.

A nonproper functional is $E(x)=+\infty$ for every $x$. Its effective domain is empty, so it has no finite-valued global minimizer. It nevertheless has coercivity and is sequentially lower semicontinuous. If a minimizer is instead defined only by $E(x)=\inf E$ without requiring finiteness, every point formally minimizes this function; under that convention the requested nonproper counterexample is impossible with the given codomain.

A noncoercive functional is $E(x)=e^x$. It is a proper extended-real function and is continuous, but its infimum zero is approached as $x\to -\infty$ and is never attained. Hence it has no global minimizer.

A failure of sequential lower semicontinuity isIt is a proper extended-real function with coercivity, but $E(1/n)\to 0<E(0)$. Its infimum is zero, while every function value is positive. Thus it has no global minimizer.

The direct method in the calculus of variations gives the following existence theorem. Suppose bounded sequences in the Banach space $\mathcal{U}$ have $\tau$-convergent subsequences, and $E$ is a proper extended-real function with coercivity and is $\tau$-sequentially lower semicontinuous. Then $E$ has a finite-valued global minimizer.

To prove this, properness makes $m=\inf E<+\infty$. First rule out $m=-\infty$: a sequence with $E(u_n)\to -\infty$ eventually lies in a fixed sublevel set, which is bounded by coercivity. A $\tau$-convergent subsequence would then have a limit $u$ with $E(u)\le -\infty$, contradicting the codomain. Hence $m$ is finite.

Choose a minimizing sequence with $E(u_n)\to m$. It eventually belongs to the bounded sublevel set $\{E\le m+1\}$. Extract $u_{n_j}\stackrel{\tau }{\to }\bar{u}$. By sequential lower semicontinuity,Thus $E(\bar{u})=m$. In particular, a reflexive Banach space with the weak topology supplies the required subsequence property by weak sequential compactness of bounded sequences in a reflexive Banach space. A strictly convex function has at most one minimizer; this is an additional property, not part of the existence theorem.

The small-parameter limit depends on data compatibility with the regularizer domain. SetFor every such $v$, optimality givesTaking the upper limit as $\alpha \downarrow 0$ and then the infimum over $v$ proves ${\Phi }_{\alpha ,f}(u_{\alpha })\to m$. Its two nonnegative contributions above $m$ must vanish, yielding the general resultIn particular, the requested zero-misfit limit holds when $f\in \overline{K(\mathrm{dom}J)}$. A sufficient condition is an exact solution $v$ with $Kv=f$ and $J(v)<\infty$; thenMembership of $f$ in $\mathcal{R}(K)$ alone is insufficient. Take $\mathcal{U}=\mathcal{V}=\mathbb{R}$, $K=I$, $f=1$, and $J$ the indicator functional of a constraint set $\{0\}$. All positive-parameter objectives are proper extended-real functions with coercivity and sequential lower semicontinuity, with the unique minimizer $u_{\alpha }=0$, yet $D(u_{\alpha })=1/2$ for every $\alpha$. Thus the first printed limit needs compatibility with the regulariser's effective domain; the second limit remains true under the given assumptions.

For $u\in L^1(\Omega )$, the total variation seminorm on a domain isUsing $C_c^1$ instead gives the same definition. The zero test field shows nonnegativity, and $\mathrm{TV}(0)=0$, so this is a proper extended-real function. Each test field defines a linear functional of $u$; taking their supremum proves convexity, equivalentlyIt is not strictly convex: two distinct constant functions both have zero total variation, as does every convex combination of them.

The function of bounded variation on a domain space isTotal variation is not coercive on $BV(\Omega )$: for $u_k\equiv k$ on a domain of positive measure, $\Vert u_k{\Vert }_{BV}=k|\Omega |\to \infty$, whereas $\mathrm{TV}(u_k)=0$. This is noncoercivity of total variation on constants; a mean-zero constraint can remove the constant obstruction.

Fix $x$, and write $p=x-\alpha G_{\alpha }(x)$ and $g=G_{\alpha }(x)$. The supplied subgradient relation gives $v=v_{\alpha }\in \lambda \partial \psi (p)$ with $g=\nabla f(x)+v$. The evaluation point in $\nabla f$ is $x$, whereas $v$ is a subgradient at the new point $p$.

Apply the subgradient inequality at $p$, with comparison point $x$:The descent lemma for the $L$-Lipschitz gradient of $f$ givesAdding these estimates proves the stronger, explicit inequalityIn particular, the objective function decreases for $0<\alpha <2/L$ whenever the proximal gradient mapping is nonzero.

To justify the requested big O notation, we must check that $g$, which depends on $\alpha$, stays bounded. Suppose $\partial (\lambda \psi )(x)$ is nonempty, and choose a fixed $w$ in this subdifferential. This holds for a finite-valued convex function on ${\mathbb{R}}^n$, including both norms in parts (a) and (b). By monotonicity of a convex subdifferential,Substituting $p-x=-\alpha g$ and $v=g-\nabla f(x)$, and then using the Cauchy-Schwarz inequality, yieldsThus $\Vert g\Vert \le C_x:=\Vert \nabla f(x)+w\Vert$, uniformly in $\alpha >0$. This local boundedness of the proximal gradient mapping makes the remainder at most $L{C}_x^2{\alpha }^2/2$, soNo differentiability of $\psi$ is needed. If an proper extended-real function is allowed as the regularization term, the subdifferential condition at $x$ is a qualification for this uniform remainder; the preceding explicit descent inequality remains the useful statement whenever its quantities are finite.

A proper convex function is both a convex function and a proper extended-real function. For example, the total variation seminorm on a domain is nonnegative, finite at zero, and a supremum of linear functionals, making it proper and convex.