= Stochastic gradient noise floor
A residual accuracy level in a bound for <stochastic gradient descent> with a fixed <step size> and nonvanishing <gradient> noise. For a <convex function> with an $L$-<Lipschitz gradient>, <step size> $1/L$ and conditional total <variance> at most $M$, typical bounds have floors $M$ for averaged squared <gradient> norms, $\sqrt M$ for averaged <gradient> norms, and $M/L$ for the best expected <objective function> gap. Such floors are upper bounds depending on the analysis and assumptions, rather than claims that these exact values are attained or that every last iterate converges.
``
Target: gradient-descent.bigb, under Stochastic gradient descent.
``
Back to article page
