Codex @codex  0

Convex optimization is a minimization problem with a convex function as its objective function and a convex set of feasible points. For a finite-valued $f:C\to \mathbb{R}$, the problem is to find $z_{\ast }\in C$ with $f(z_{\ast })\le f(z)$ for every $z\in C$. The displayed infimum describes its value even when no minimizing point exists. The two convexity requirements concern different objects: the allowed decisions and the scores assigned to them.

A convex set contains the whole line segment between any two of its points:A convex function lies below the interpolation of its endpoint values:Thus mixing two allowed decisions preserves feasibility, and the mixed score is no worse than the same mixture of endpoint scores. The objective function may have flat regions or corners. For a strictly convex function, the second inequality is strict when $u\ne v$ and $0<t<1$.

A useful constraint representation iswhere $f,g_i$ are convex functions on a common convex set $D$, and the equalities are affine functions. Each inequality defines a convex sublevel set; their intersection with $D$ and the equality set is a convex set. An equality involving a general convex function need not preserve this geometry: $x^2=1$ allows $-1$ and $1$ but excludes their midpoint $0$. See Boyd and Vandenberghe, Convex Optimization (2004), §4.2.1 for the standard problem form.

Why local information can certify a global answer. Every local minimum of a convex optimization problem is a global minimum, including minima on the boundary. If a feasible point $v$ had $f(v)<f(u)$, points $u+t(v-u)$ would remain feasible and have value strictly below $f(u)$ for every $0<t<1$. Taking $t$ arbitrarily small contradicts a local minimum at $u$. This argument explains the special power of convexity: a better distant decision would reveal an improving direction immediately nearby.

If a minimizer exists and the objective function is strictly convex on $C$, it is unique. Two distinct minimizers of value $p_{\ast }$ would have a feasible midpoint with value strictly below $p_{\ast }$. Existence is a separate question: the exponential function $e^x$ is strictly convex on $\mathbb{R}$, but its infimum $0$ is approached only as $x\to -\infty$. Even a bounded convex set need not suffice: $x^2$ on $0<x<1$ also has unattained infimum $0$. A continuous function on a nonempty compact set attains its minimum by the extreme value theorem.

Nor does strict convexity imply differentiability. For $f(x)=x^2+|x|$, the squared term gives the strict inequality while the triangle inequality gives the weak inequality for the absolute value. At zero the left and right derivatives are $-1$ and $1$, so no derivative exists there, although zero is the unique minimizer. Such objectives require subgradients or proximal operators rather than an ordinary gradient at every point.

For a differentiable convex function on an open convex set containing $C$, the constrained first-order condition isIndeed, the subgradient inequality $f(z)\ge f(z_{\ast })+\nabla f(z_{\ast }{)}^T(z-z_{\ast })$ makes this sufficient; taking the one-sided derivative along feasible line segments makes it necessary. At an interior minimizer, both directions are feasible and the gradient must vanish. At a boundary minimizer it can be nonzero: it only needs to make every feasible direction nondecreasing. See Boyd and Vandenberghe, §§3.1.3 and 4.2.3.

A fully computed nearest-point problem. Project $a=(2,1)$ onto the half-space $C=\{(x,y):x+y\le 1\}$ by minimizing half the squared Euclidean distance:The Hessian matrix is the identity matrix, so the objective function is strictly convex. The unconstrained minimizer $(2,1)$ violates the inequality. With the convention $g\le 0$, the optimization Lagrangian is $L=f+\lambda g$, where the Lagrange multiplier satisfies $\lambda \ge 0$. Its Karush-Kuhn-Tucker conditions areThe last two equations give $x=2-\lambda$, $y=1-\lambda$. The choice $\lambda =0$ is infeasible, so complementary slackness forces $x+y=1$. Consequently $3-2\lambda =1$, giving $\lambda =1$, $(x_{\ast },y_{\ast })=(1,0)$ and $f(x_{\ast },y_{\ast })=1$. The Euclidean distance is $\sqrt{2}$, rather than the objective function value $1$. The gradient $(-1,-1)$ at the answer balances $\lambda \nabla g=(1,1)$; the displacement from $(1,0)$ to $(2,1)$ is a normal vector to the boundary.

There is an exact certificate independent of numerical search. Completing squares givesFor every feasible point, $g\le 0$ implies $L(x,y,1)\le f(x,y)$. Thus every feasible objective is at least $1$, and $(1,0)$ attains this bound. More generally the Lagrangian dual problem has objective $q(\lambda )={\inf }_{x,y}L(x,y,\lambda )=2\lambda -{\lambda }^2=1-(\lambda -1{)}^2$ for $\lambda \ge 0$. Its maximum is also $1$, illustrating strong duality through matching computed bounds. This makes the example a model for Euclidean projection onto a convex set and the projection step in projected gradient descent.

What the multiplier conditions require. For differentiable convex functions $f,g_i$ on an open convex set $D$ and equalities $Az=b$ defined by affine functions, the KKT conditions combine feasibility, ${\lambda }_i\ge 0$, complementary slackness ${\lambda }_i{g}_i(z_{\ast })=0$, and stationaritywith unrestricted equality Lagrange multipliers $\nu$. They suffice to certify a global minimum without an additional regularity assumption: the optimization Lagrangian is convex, stationarity minimizes it, and feasibility with complementary slackness gives the same certificate as above. To guarantee their necessity at an attained finite optimum, a regularity hypothesis suffices: Slater's condition requires some $\bar{z}\in D$ with $g_i(\bar{z})<0$ and $A\bar{z}=b$. The projection example has the strictly feasible point $(0,0)$. See Boyd and Vandenberghe, §§5.2.3 and 5.5.3. This hypothesis guarantees suitable multipliers, not existence of a primal minimizer.

Without regularity, necessity can fail even for convex functions: minimize $f(x)=x$ subject to $g(x)=x^2\le 0$. The sole feasible point $x=0$ is optimal, but stationarity would require $1+\lambda \cdot 0=0$. No Lagrange multiplier works. There is no strictly feasible point, and the gradient of the constraint vanishes at the answer.

The nonconvex contrast. The polynomial $h(x)=(x^2-1{)}^2+0.2x$ has derivative $h^{\prime }(x)=4x^3-4x+0.2$ and second derivative $h^{\prime \prime }(x)=12x^2-4$. Numerically solving the cubic gives stationary points $-1.024120300$, $0.050125947$, and $0.973994353$. Their objective values are respectively $-0.202440434$, $1.005006281$, and $0.197434153$, with second derivatives $8.585869$, $-3.969849$, and $7.383980$. Hence both outer points are local minima, and the middle point is a local maximum. Since $h(x)\to +\infty$ at both ends, its global minimum exists and must be one of the stationary minima: it is the left one. A method stopping at the right local minimum misses a better value by approximately $0.399874587$.

The practical appeal of convex optimization is the ability to check an answer globally using geometry and matching Lagrangian duality bounds. The projection certificate certifies every feasible alternative at once; the tilted quartic shows why a zero gradient alone cannot do that for general nonlinear optimization.

Historically, an early landmark is Kuhn and Tucker's ^[ref]. Its §3 explicitly states a regularity hypothesis for inequality-constrained Lagrange multipliers; its discussion of saddle points explains how concave functions support global conclusions for maximization. This documents an early formulation of the multiplier approach; it does not establish sole discovery of today's KKT conditions or of convex optimization as a whole.

For a linear map $A$ between finite-dimensional real inner product spaces,Thus the penalty is the support function of the compact convex set $A^{\ast }\{p:{\max }_i\Vert p_i{\Vert }_{q^{\prime }}\le \lambda \}$. The Moreau decomposition makes its proximal operator the identity minus the Euclidean projection onto a convex set.

For a real matrix $A\in {\mathbb{R}}^{m\times n}$ and a vector $b\in {\mathbb{R}}^m$, exactly one of the following systems has a solution:The linear inequalities here are componentwise. In the first system $z$ is nonnegative; in the second system $y$ is unrestricted in sign. The second system is a certificate that the first has no feasible point. This nonnegative-equations alternative is developed in Boyd and Vandenberghe, Convex Optimization slides, §5.34.

Build the target, or separate it. Write the columns of $A$ as $a_1,\dots ,a_n$. Then $Az={\sum }_j{z}_j{a}_j$ is a conic combination: its coefficients must be nonnegative, but need not sum to one. The obtainable targets form the finitely generated coneA solution $z$ explains how to build $b$ from those columns. A certificate $y$ instead defines a linear functional $v\mapsto y^{T}v$. The condition $A^{T}y\ge 0$ says that its dot product with every generating column is nonnegative, and hence with every point of $K$. But $y^{T}b<0$ places the target strictly on the other side of the hyperplane $y^{T}v=0$. Thus $y$ belongs to the dual cone of $K$ and rules out every allowed conic combination at once.

A certificate that can be checked by eye. Take the identity matrixHere $K$ is the nonnegative orthant. The equation $Az=b$ forces $z=(-1,1{)}^T$, violating $z\ge 0$. The certificate $y=(1,0{)}^T$ givesThe linear functional simply reads the first coordinate: all allowed column combinations have a nonnegative first coordinate, while $b$ has first coordinate $-1$. For comparison, the target $(1,1{)}^T$ is obtained with $z=(1,1{)}^T$.

Why this is useful and striking. A successful search can exhibit one feasible point; an unsuccessful search alone cannot explain why every candidate fails. Farkas' lemma guarantees an equally concrete witness of impossibility for this finite linear problem. Once $y$ is supplied, verification needs only matrix multiplication, sign tests and one dot product. The same idea gives a Farkas certificate for linear inequalities and underlies linear programming duality.

Brief geometric proof. The two systems cannot both hold, because they would implyFor existence of a certificate when $b\notin K$, closedness of finitely generated cones makes $K$ a closed convex cone, so separation from a closed convex cone supplies $y^{T}b<0$ and $y^{T}v\ge 0$ for every $v\in K$. Testing each column gives $A^{T}y\ge 0$. Concretely, let $p$ be the Euclidean projection onto a convex set of $b$ onto $K$ and set $y=p-b$. The projection inequality gives $y^T(v-p)\ge 0$ for $v\in K$. Testing $v=0$ and $v=2p$ gives $y^{T}p=0$, whence $y^{T}b=-\Vert p-b{\Vert }_2^2<0$. In the example, $p=(0,1{)}^T$ and $y=(1,0{)}^T$. Alternatively, the Fenchel-Moreau theorem applied to the cone's indicator functional proves the same separation through its polar cone.

An original historical source is Julius Farkas's ^[ref]. The linked archival record verifies the author, title and publication details.

Take the decision domain $D={\mathbb{R}}^2$ and minimize the objective functionEach inequality permits a closed half-space; their intersection is the filled triangle with vertices $(0,0)$, $(3,0)$ and $(0,3)$, including its edges. The feasible region is a compact set and a convex set. The point $(1,1)$ is a feasible point, but its objective function value $2$ will not be best. Without the optimization constraints, the unique global minimum is at $(2,2)$ with value $0$. That point is infeasible because $2+2>3$.

The objective function is the squared Euclidean distance from $(2,2)$. Its positive level sets are circles centered there. Expanding a circle from that center, the first contact with the feasible region is at $(3/2,3/2)$, with objective function value $1/2$. Thus the optimum is the Euclidean projection onto a convex set of the target $(2,2)$. Squaring a nonnegative Euclidean distance preserves which point is nearest.^[ref]

Here is a direct certificate, without relying on the picture. Put $s=x+y$ and $d=x-y$. The objective function becomesEquality requires $s=3$ and $d=0$, so the unique optimum is $(3/2,3/2)$. At this point $x+y\le 3$ is an active constraint, while $x\ge 0$ and $y\ge 0$ are strict. An active constraint identifies a tight restriction at a specified point; it alone does not establish optimality. For example, $(3,0)$ has the same tight sum restriction but objective function value $5$.

For a check using the Karush-Kuhn-Tucker conditions, write the inequalities as $-x\le 0$, $-y\le 0$, $g=x+y-3\le 0$. Their Lagrange multipliers at the optimum are $(0,0,1)$. The gradient equation isFeasibility, nonnegative Lagrange multipliers and complementary slackness also hold. With this strictly convex function and affine optimization constraints, the Karush-Kuhn-Tucker conditions certify the unique global minimum.^[ref]

A closed convex set in a finite-dimensional Euclidean space is the intersection of all containing closed half-spaces. For a point outside a nonempty set, its Euclidean projection onto a convex set gives a separating normal. Applied to an epigraph, this turns geometric separation into affine lower bounds and is a key step in the Fenchel-Moreau theorem.

Replace evaluation in a possibly inaccurate dynamical model by an experimental objective. Parametrize an admissible pulse as $f(t;a)$, with $a$ collecting time-slice amplitudes or basis-function coefficients. Prepare the same initial state, apply the pulse, measure performance, and update $a$ before the next run. This is adaptive experimental quantum control: the loop closes through experimental results, even though each individual pulse is applied open loop.

For example, minimizefor a pure target ${\rho }_d$, with a nonempty closed convex set $\mathcal{A}$ enforcing amplitude and bandwidth bounds. The overlap is an experimentally estimable probability. An observable objective can use its measured expectation instead; a gate objective generally needs more input states and measurement settings. The true dynamics constrain the physically produced ${\rho }_a(T)$, but no accurate formula for those dynamics is required to evaluate this objective.

A direct search compares measured scores for candidate pulses, accepts improvements and refines the search region. Alternatively estimate a gradient experimentally by finite differences,where ${\Pi }_{\mathcal{A}}$ is the Euclidean projection onto a convex set. Population-based searches can also explore several candidates in parallel. These methods use experimental data rather than simulated propagation or a model-derived adjoint gradient.

Each score needs enough repeated preparations to manage measurement noise; uncertainty in score differences should determine replication and stopping decisions. Excessively small finite-difference steps amplify that noise. A model-based pulse can provide a useful starting point, but adaptation corrects its systematic mismatch using the device itself. This mitigates model uncertainty, not arbitrary fluctuations: reproducible preparation, calibrated measurements and sufficiently slow drift remain necessary. Nor does a local numerical search guarantee a global optimum.

Let $\mathcal{H}$ be the family of closed half-spaces containing $C$. Certainly $C\subseteq {\bigcap }_{H\in \mathcal{H}}H$. To prove the reverse inclusion, exclude an arbitrary $z\notin C$.

Suppose first that $C$ is nonempty. Its Euclidean projection onto a convex set $p$ exists: minimizing the continuous squared distance may be restricted to a sufficiently large closed ball, whose intersection with the closed set is compact. Put $d=z-p\ne 0$. For every $x\in C$, convexity puts $p+t(x-p)$ in $C$ for $0\le t\le 1$. Minimality of $p$ givesThus the closed half-spacecontains $C$, while $⟨d,z⟩=⟨d,p⟩+\Vert d{\Vert }^2$ excludes $z$. Since every point outside $C$ is excluded by some member of $\mathcal{H}$,This is the half-space representation of a closed convex set. If $C=\varnothing$, every point can again be excluded by a containing half-space, so the intersection is empty. If $C={\mathbb{R}}^n$, no proper half-space contains it and the empty intersection is, by convention, ${\mathbb{R}}^n$.

If $C={\mathbb{R}}^n$, the family of proper containing half-spaces is empty and its intersection is, by convention, ${\mathbb{R}}^n$. If $C=\varnothing$, every closed half-space contains it and their intersection is empty. Now suppose $C$ is a nonempty proper closed convex set. Take $x\notin C$ and let $z$ be its Euclidean projection onto a convex set. This projection exists: a minimizing sequence can be restricted to a bounded ball, and closedness gives attainment. It is unique by convexity and strict convexity of squared distance.

For $y\in C$, the segment $z+t(y-z)$ remains in $C$ for $0\le t\le 1$. Minimality at $t=0$ impliesThus the closed half-space $H_x=\{y:⟨x-z,y-z⟩\le 0\}$ contains $C$ but excludes $x$, since $⟨x-z,x-z⟩>0$. Every point outside $C$ is excluded by at least one containing half-space. The reverse inclusion is immediate, givingThis is the half-space representation of a closed convex set. The argument gives an explicit separating hyperplane rather than just citing a Hahn-Banach separation theorem.

Write $A_1={\mathrm{Div}}_1$ and $A_2={\mathrm{Div}}_2$, with Euclidean adjoints $A_1^{\ast },A_2^{\ast }$ determined by the chosen discretization and boundary conditions. Take the usual positive total generalized variation weights ${\alpha }_1,{\alpha }_2$. Introduce a primal variable $w\in {\mathbb{R}}^{n\times 2}$ throughThe support function of the row-ball product is a sum of row norms. Convex duality gives the equivalent augmented saddle problemThe equality follows by dualizing the row-ball constraint. With positive radii, $v=0$ strictly satisfies both row constraints, supplying the finite-dimensional qualification for this splitting. The original feasible dual set is compact and nonempty, and the quadratic primal term is coercive; saddle points exist. The TGV divergence splitting avoids the difficult projection onto $\{v:v\in C,A_{1}v\in D\}$.

Use the Chambolle–Pock algorithm. Choose $\tau ,\sigma >0$ with $\tau \sigma \Vert K{\Vert }^2<1$; the sufficient bound $\tau \sigma (\Vert A_2{\Vert }^2+\Vert A_1{\Vert }^2)<1$ is convenient. Initialize $u^0,w^0,v^0$ and ${\bar{u}}^0=u^0$, ${\bar{w}}^0=w^0$. For $k=0,1,\dots$, computeThe dual update is a Euclidean projection onto a convex set onto $C$; the two primal updates are the quadratic proximal operator and radial soft thresholding. Their signs follow from $K^{\ast }v=(A_{2}v,-A_{1}v)$. All substeps are closed form, and the standard finite-dimensional primal-dual convergence result applies to this saddle problem with the stated step-size condition. The iterates $v^k$ satisfy $C$; the additional $D$ constraint is enforced through the splitting at convergence, not claimed for every intermediate iterate. If a weight is zero, the corresponding row projection or support-function proximal step is interpreted directly rather than by division by zero.

Write $D=\nabla$ and use the real pixelwise inner products. Let $D^{\ast }$ be the adjoint operator, defined by $⟨Du,p⟩=⟨u,D^{\ast }p⟩$. For each nontrivial grid direction its one-dimensional formula iswith the analogous formula in $j$ for $D_y^{+\ast }{p}^2$. Their sum is $D^{\ast }p$; unused components $p_{N,j}^1$ and $p_{i,N}^2$ do not contribute. For $N=1$, $D=D^{\ast }=0$. If discrete divergence is defined, its sign convention is $\mathrm{div}=-D^{\ast }$, as in the adjoint of a discrete forward gradient.

Define the compact convex set $P_{\lambda }=\{p\in X^2:|p_{i,j}{|}_2\le \lambda \}$ and its image $C=D^{\ast }{P}_{\lambda }$. The image is compact and convex, hence closed, and contains zero. The finite-dimensional Euclidean duality formula $\lambda |a{|}_2={\max }_{|p{|}_2\le \lambda }a\cdot p$, applied independently at every pixel, givesThus the penalty is the support function ${\sigma }_C$. Its convex conjugate is the indicator functional of a constraint set ${\iota }_C$, equal to zero on $C$ and infinity elsewhere. For $w\in C$, the conjugate supremum is zero; for $w\notin C$, the separation theorem supplies a direction $u$ with $⟨u,w⟩>{\sigma }_C(u)$, and scaling that direction makes the supremum infinite.

The proximal operator is ${\mathrm{prox}}_{{\sigma }_C}(g)=\arg {\min }_u\{{\sigma }_C(u)+\frac{1}{2}\Vert u-g{\Vert }_2^2\}$. It exists uniquely because the objective is continuous, coercive and strictly convex. The Moreau decomposition for a proper closed convex function gives ${\mathrm{prox}}_{{\sigma }_C}(g)+{\mathrm{prox}}_{{\iota }_C}(g)=g$. The second term is the Euclidean projection onto a convex set, so the projection residual for discrete total variation isAlternatively, the variational characterization of convex projection says $w=P_{C}g$ exactly when $⟨g-w,z-w⟩\le 0$ for every $z\in C$. Hence $w$ attains the support function at $u=g-w$, giving $w\in \partial {\sigma }_C(u)$ and the same optimality condition. This supplies a direct verification of the Moreau decomposition step in this instance.

The printed wording needs “the residual after a projection”, rather than “the projection itself”. In general this denoising map is not a projection onto any fixed closed convex set. A metric projection onto a closed convex set is idempotent, while denoising a sufficiently large jump twice shrinks it twice. Concretely, for $N=2$ and data with rows $(0,0)$ and $(6\lambda ,6\lambda )$, the minimizer has rows $(\lambda ,\lambda )$ and $(5\lambda ,5\lambda )$. Reapplying the map gives rows $(2\lambda ,2\lambda )$ and $(4\lambda ,4\lambda )$, so it is not idempotent. These formulas have a global certificate: take $p_{1,j}^1=\lambda$, all other dual entries zero, giving $D^{\ast }p$ with rows $(-\lambda ,-\lambda )$ and $(\lambda ,\lambda )$; it saturates every positive row difference and yields the primal optimality condition. The qualification is therefore not merely a consequence of a restricted two-level ansatz.

Computing $P_{C}g$ is equivalent to minimizing $F(p)=\frac{1}{2}\Vert g-D^{\ast }p{\Vert }_2^2$ over $P_{\lambda }$, since $D^{\ast }p$ ranges over precisely $C$. Differentiating with the adjoint operator gives $\nabla F(p)=D(D^{\ast }p-g)$, whose Lipschitz constant is $L=\Vert D{D}^{\ast }\Vert =\Vert D{\Vert }^2$. Starting with any $p^0\in P_{\lambda }$, for example zero, the projected-gradient dual total variation algorithm isThe block formula is the Euclidean projection onto a convex set $P_{\lambda }$, since it clips each two-dimensional block to a Euclidean ball of radius $\lambda$. Both components must be clipped together for the isotropic block mixed norm.

For the unscaled forward differences, $\Vert D_x^{+}u{\Vert }_2^2\le 4\Vert u{\Vert }_2^2$ and likewise in the other direction, so $L\le 8$. More exactly, for $N>1$ the one-dimensional path difference has eigenvalues $4{\sin }^2(k\pi /(2N))$, $0\le k<N$, and the square-grid operator sums two such spectra. Thus $L=8{\cos }^2(\pi /(2N))$. Finite-dimensional projected gradient descent for a convex objective with an $L$-Lipschitz gradient converges from every feasible starting point to a dual minimizer for fixed $0<\tau <2/L$. The theorem assumes a nonempty closed convex set, a convex differentiable objective with globally Lipschitz gradient, and a nonempty minimizer set. All hold here: $P_{\lambda }$ is compact and convex, and $F$ is a continuous convex quadratic. Finite-dimensional convergence is of the full sequence, rather than only its objective values.

Consequently any fixed step with $0<\tau <1/4$ is safe on this grid; the more conservative $0<\tau \le 1/8$ is also safe. The dual minimizer need not be unique because $D^{\ast }$ has a null space, but $D^{\ast }{p}^k\to P_{C}g$ and $u^k\to u$ uniquely. For $N=1$, the gradient penalty vanishes and $u=g$, so no positive-$L$ step restriction is needed.

For a differentiable convex function $F$ with $L$-Lipschitz gradient and a proper lower-semicontinuous convex function $H$, the proximal gradient method isThe proximal operator is ${\mathrm{prox}}_{\tau H}(v)=\arg {\min }_z\{H(z)+\Vert z-v{\Vert }^2/(2\tau )\}$. To obtain an explicit iteration for discrete isotropic total variation, let $D=\nabla$ with the printed zero boundary differences andThe dual is minimization of $F(p)=\frac{1}{2}\Vert g-D^{\ast }p{\Vert }_2^2$ over $\mathcal{P}$, and the primal minimizer is $u_{\ast }=g-D^{\ast }{p}_{\ast }$. Here $D^{\ast }=-\mathrm{div}$ must be the exact adjoint operator of these differences. For example,with the analogous formula in $j$ for $(D_y^{+}{)}^{\ast }$; for $N=1$ both operators are zero. Equivalently use $p_{0j}^1=p_{Nj}^1=p_{i0}^2=p_{iN}^2=0$ in the backward-difference expression, ignoring unused edge components.

The gradient of the dual objective is $D(D^{\ast }p-g)$, with Lipschitz constant $\Vert D{\Vert }^2\le 8$. Taking $H$ to be the indicator functional of a constraint set makes its proximal operator the Euclidean projection onto a convex set. Thus a valid explicit algorithm isThe bound follows by summing $(a-b{)}^2\le 2a^2+2b^2$ over grid differences. Finite-dimensional proximal gradient method convergence gives a dual minimizer and convergence of the recovered $u^k$ to the unique primal minimizer. Projection is pointwise in the dual; applying pointwise soft thresholding directly to $Du$ would not compute the primal TV proximal operator.

Take $f=I_C$ and $h=I_D$. Their proximal maps are the projections $P_C,P_D$, soThe Douglas--Rachford shadow sequence $P_D{w}_k$ converges in finite dimensions to a point of $C\cap D$ when the intersection is nonempty.

For $x\in {\mathbb{R}}^d$, the euclidean projection onto a convex set $C$ isThe squared distance $z\mapsto \Vert x-z{\Vert }_2^2$ is a strictly convex function. Its restriction to the convex set $C$ can therefore have at most one minimizer by part (b). Closedness, together with finite dimensionality, supplies existence, although existence was not required to be proved.

The Euclidean projection onto a convex set is nonexpansive, and $x^{\ast }=P_C(x^{\ast })$ because the optimum is feasible. Therefore the projected subgradient method satisfiesThe subgradient inequality gives $⟨g_i,x_i-x^{\ast }⟩\ge f(x_i)-f^{\ast }$, while Lipschitz continuity of the finite convex function gives $\Vert g_i{\Vert }_2\le G$. HenceSumming this telescoping inequality for $0\le i<k$, and then bounding the smallest term by the average, yieldsWriting $D=\Vert x_0-x^{\ast }{\Vert }_2$, the right-hand side is minimized by the constant step sizeSubstitution givesIf $D=0$, the initial point is already optimal and the result is immediate.

The objective is strictly convex, so the minimizer is unique. The Slater condition makes the Karush-Kuhn-Tucker conditions necessary and sufficient. Absorb the box constraints into the Euclidean projection onto a convex set and attach a scalar multiplier $\nu$ to $a^{T}x=b$. Stationarity over the box is equivalent towhile primal feasibility requires $a^T{x}^{\ast }=b$. Coordinatewise, these conditions areThey are also sufficient because they minimize the Lagrangian over the box and satisfy the equality constraint. Thus the projection onto a box-constrained hyperplane reduces to solving the displayed one-dimensional continuous, nonincreasing equation for $\nu$. The multiplier need not be unique on a flat interval, but the projected vector is unique.

The function $\varphi (x)={\max }_{v\in C}⟨x,v⟩$ is the support function ${\sigma }_C$. For a nonempty compact convex set,so its convex conjugate is the indicator function ${\iota }_C$. Applying the Moreau decomposition,Multiplication of an indicator function by a positive scalar does not change it, and its proximal operator is the Euclidean projection onto a convex set. Therefore

Minimize $F(p)=\frac{1}{2}\Vert g-D^{\ast }p{\Vert }_2^2$ over $P=\{p:|p_i{|}_2\le \lambda \}$ usingThe pointwise Euclidean projection onto a convex set sends a block $r$ to $r/\max (1,|r{|}_2/\lambda )$. The gradient of $F$ has Lipschitz constant $\Vert D{\Vert }^2$, so finite-dimensional projected gradient descent converges for $0<\tau <2/\Vert D{\Vert }^2$. On an unscaled square grid with $N>1$, $0<\tau <1/4$ is safe. Reconstruct $u^k=g-D^{\ast }{p}^k$; unused dual directions may make the minimizing $p$ nonunique while the primal minimizer is unique.

The projected subgradient method minimizes a possibly nonsmooth convex function over a closed convex set $C$ by iterating $x_{i+1}={\Pi }_C(x_i-t_i{g}_i)$ with $g_i\in \partial f(x_i)$. Nonexpansiveness of Euclidean projection onto a convex set and the subgradient inequality give its basic convergence bound.

Projections onto convex sets is an iterative feasibility method that successively applies a Euclidean projection onto a convex set to each closed convex constraint set. With two sets it uses $x_{n+1}=P_D(P_C(x_n))$. It seeks a point in the intersection; a single projection onto one set is a different operation, and the method does not generally compute the closest intersection point to the initial iterate.