Alternating proximal-gradient operator Created 2026-10-06 Updated 2026-11-05
Let be a differentiable convex function whose full gradient is -Lipschitz continuous, with . The alternating update first solves the implicit step and then the explicit step:
The implicit step is the proximal operator of , so it exists uniquely. The gradient bound must control the full joint variable; smoothness alone supplies no global constant .
For two updates, let be the difference of their mixed points and the corresponding full gradient difference. The input and output differences are and . The mixed-point norm identity for alternating updates and Baillon–Haddad theorem give
Hence is a -averaged operator for . Its fixed points have both block gradients zero and are exactly the global minimizers of . If a minimizer exists, Browder convergence theorem for averaged operators gives norm convergence of iteration in the stated finite-dimensional setting.
Let be convex with an -Lipschitz gradient, , and let be a proper lower semicontinuous convex function. For , take
The proper closed convex assumption on ensures that the implicit section has a well-defined proximal operator.
For two mixed points with difference , write for the gradient difference of and for the difference of the selected subgradients of . Inputs differ by and outputs by . Monotonicity of a convex subdifferential gives , while cocoercivity gives . Expanding these squared norms proves
for . Thus the combined update is firmly nonexpansive. Its fixed points are the minimizers of ; if one exists, the Browder convergence theorem for averaged operators gives norm convergence in finite dimensions.
The Browder convergence theorem for averaged operators states: if is an averaged operator and its fixed-point set is nonempty, then for every starting point, converges in norm to a fixed point of . The theorem assumes a fixed point exists; averaging by itself does not create one.
The finite-dimensional mechanism is worth making explicit. For any fixed point , the averaged-operator inequality gives
This is Fejér monotonicity with respect to the fixed-point set. It makes the sequence bounded and, by summation, makes finite. Hence its residual tends to zero. Choose a convergent subsequence ; continuity of the nonexpansive mapping implies . Apply Fejér monotonicity with . The nonincreasing distances have a subsequence tending to zero, so the whole sequence converges to . This last compactness step uses finite dimension; a corresponding general Hilbert space statement usually gives weak convergence.
Assume , as is necessary for convergence to a minimizer. The printed hypotheses alone omit this condition: in scalar is convex with an -Lipschitz gradient for any prescribed , but has no minimizer, and the update sends to .
Under existence, define the full update . The proximal -step is uniquely defined by strong convexity of its quadratic term and satisfies
The partial subdifferential in this equation belongs to the scalar function ; differentiating a subdifferential itself would not define the printed proximal minimization.
We prove directly that the alternating proximal-gradient operator is averaged. For two inputs and , set
The input difference is , the output difference is , and the difference of the two update residuals is . Exact expansion gives
The first inequality uses cocoercivity at the mixed points . Therefore is -averaged for the requested . In particular, the endpoint is covered: is firmly nonexpansive, not merely nonexpansive.
A fixed point has and , so the two optimality equations give and . Conversely, at a minimizer both gradients vanish and neither step moves. By convexity,
The assumed existence of a minimizer supplies a fixed point. The Browder convergence theorem for averaged operators now gives
No uniqueness of the minimizer is needed. The same estimate even proves averagedness for in this smooth setting, but the requested interval already suffices. The existence condition and the use of the gradient at are both essential to the proof as presented.