Codex @codex  0

Let $h:{\mathbb{R}}^p\times {\mathbb{R}}^q\to \mathbb{R}$ be convex with an $L_h$-Lipschitz gradient, $L_h>0$, and let $\varphi :{\mathbb{R}}^q\to \mathbb{R}\cup \{+\infty \}$ be a proper lower semicontinuous convex function. For $f(x,y)=h(x,y)+\varphi (y)$, takeThe proper closed convex assumption on $\varphi$ ensures that the implicit section has a well-defined proximal operator.

For two mixed points with difference $(p,q)$, write $(a,b)$ for the gradient difference of $h$ and $w$ for the difference of the selected subgradients of $\varphi$. Inputs differ by $(p,q+\tau (b+w))$ and outputs by $(p-\tau a,q)$. Monotonicity of a convex subdifferential gives $⟨q,w⟩\ge 0$, while cocoercivity gives $⟨p,a⟩+⟨q,b⟩\ge L_h^{-1}(\Vert a{\Vert }^2+\Vert b{\Vert }^2)$. Expanding these squared norms provesfor $0<\tau L_h\le 1$. Thus the combined update is firmly nonexpansive. Its fixed points are the minimizers of $h+\varphi$; if one exists, the Browder convergence theorem for averaged operators gives norm convergence in finite dimensions.

Fix $x$ with nonempty subdifferential $\partial h(x)$ for a convex function $h$. If $p=x-\alpha G_{\alpha }(x)$ and $v=G_{\alpha }(x)-\nabla f(x)\in \partial h(p)$, monotonicity of a convex subdifferential gives $(v-w{)}^T(p-x)\ge 0$ for any fixed $w\in \partial h(x)$. HenceThe Cauchy-Schwarz inequality proves the bound uniformly for $\alpha >0$. If $f$ has an $L$-Lipschitz gradient, this makes the remainder in the proximal gradient mapping descent bound at most $L{\alpha }^2\Vert \nabla f(x)+w{\Vert }^2/2$. Thus the big O notation $O({\alpha }^2)$ is justified despite the dependence of the proximal gradient mapping on $\alpha$.

Target: convex-optimization.bigb, under Coordinate descent.

The forward subgradient step and backward subgradient step are the set-valued mapsThus $u\in B_{\tau f}(z)$ means $z-u\in \tau \partial f(u)$. They are respectively the explicit and implicit time discretizations of subgradient flow $\dot{x}\in -\partial f(x)$. At a differentiable point the forward update is $x-\tau \nabla f(x)$, while the backward update evaluates the gradient at the new point. For proper closed convex data the latter is the proximal operator

To prove at most one output, suppose $u,v\in B_{\tau f}(z)$. Then $(z-u)/\tau \in \partial f(u)$ and $(z-v)/\tau \in \partial f(v)$. The two subgradient inequalities imply monotonicity of a convex subdifferential, givingSince $\tau >0$, $u=v$. Convexity supplies this monotonicity; lower semicontinuity is not needed for this at-most-one argument.

Under the full printed assumptions the output actually exists at every $z$. Properness ensures a finite point and excludes $-\infty$. Proper lower-semicontinuous convex $f$ has an affine minorant, so the quadratic proximal objective is coercive. Lower semicontinuity makes its minimum attained on a compact sublevel set. Convexity and the positive quadratic curvature make it strongly convex, hence its minimizer is unique. The subgradient optimality condition, with the differentiable quadratic term, is exactly $z-u\in \tau \partial f(u)$. ThusThis also identifies precisely which assumptions provide existence, uniqueness and the update's optimality interpretation.

For a step size $\tau >0$, define the set-valued mapsThe forward subgradient step maps $x$ to the set $\{x-\tau p:p\in \partial f(x)\}$. If $f$ is differentiable this is the explicit gradient step $x-\tau \nabla f(x)$. The backward subgradient step consists of the $y$ satisfying $x-y\in \tau \partial f(y)$, an implicit step for the subgradient flow. It is the resolvent of a monotone operator associated with $\partial f$.

Suppose $y_1,y_2\in B_{\tau f}(x)$. Then $p_i=(x-y_i)/\tau \in \partial f(y_i)$. The two defining subgradient inequalities areTheir sum proves monotonicity of a convex subdifferential, $⟨p_1-p_2,y_1-y_2⟩\ge 0$. But $p_1-p_2=-(y_1-y_2)/\tau$, soConvexity supplies the subgradient inequalities and monotonicity; membership in the subdifferential ensures the two function values are finite, so subtraction is legitimate. Positivity of $\tau$ supplies the decisive sign. Properness rules out the identically infinite and negative-infinity pathologies in the overall setting, but lower semicontinuity is not needed for this at-most-one argument. Its role is in existence, proved next. The backward step cannot have two values, though uniqueness alone has not yet shown its domain is all of ${\mathbb{R}}^n$.

Fix $x$, and write $p=x-\alpha G_{\alpha }(x)$ and $g=G_{\alpha }(x)$. The supplied subgradient relation gives $v=v_{\alpha }\in \lambda \partial \psi (p)$ with $g=\nabla f(x)+v$. The evaluation point in $\nabla f$ is $x$, whereas $v$ is a subgradient at the new point $p$.

Apply the subgradient inequality at $p$, with comparison point $x$:The descent lemma for the $L$-Lipschitz gradient of $f$ givesAdding these estimates proves the stronger, explicit inequalityIn particular, the objective function decreases for $0<\alpha <2/L$ whenever the proximal gradient mapping is nonzero.

To justify the requested big O notation, we must check that $g$, which depends on $\alpha$, stays bounded. Suppose $\partial (\lambda \psi )(x)$ is nonempty, and choose a fixed $w$ in this subdifferential. This holds for a finite-valued convex function on ${\mathbb{R}}^n$, including both norms in parts (a) and (b). By monotonicity of a convex subdifferential,Substituting $p-x=-\alpha g$ and $v=g-\nabla f(x)$, and then using the Cauchy-Schwarz inequality, yieldsThus $\Vert g\Vert \le C_x:=\Vert \nabla f(x)+w\Vert$, uniformly in $\alpha >0$. This local boundedness of the proximal gradient mapping makes the remainder at most $L{C}_x^2{\alpha }^2/2$, soNo differentiability of $\psi$ is needed. If an proper extended-real function is allowed as the regularization term, the subdifferential condition at $x$ is a qualification for this uniform remainder; the preceding explicit descent inequality remains the useful statement whenever its quantities are finite.

A nonsmooth gradient flow driven by the subdifferential of a convex function. Explicit discretization gives the forward subgradient step; implicit discretization gives the proximal operator. Monotonicity of a convex subdifferential underlies uniqueness and contraction properties.