Projected stochastic gradient descent combines a random gradient step with the Euclidean projection onto a convex set:In a finite-dimensional Euclidean space, a nonempty closed convex set has a unique projection. This ensures that the iteration is well defined and feasible. For the scalar constraint , the update is simply .
For every feasible , projection is nonexpansive and fixes , so . This inequality underlies convergence estimates after taking conditional expectations. On a general closed nonconvex feasible set, nearest points may not be unique and this convex projection argument does not apply.
Articles by others on the same topic
There are currently no matching articles.