Stochastic gradient second-moment identity ($\mathbb{E}[\Vert H{\Vert }^2\mid X]=\Vert \nabla f(X){\Vert }^2+\mathbb{E}[\Vert H-\nabla f(X){\Vert }^2\mid X]$)

For an unbiased stochastic gradient $H$ at $X$ with finite conditional second moment, expand $H=\nabla f(X)+(H-\nabla f(X))$ in the squared Euclidean norm. The cross term has zero conditional expectation, which proves the displayed identity. It separates the squared true gradient from the total conditional variance of its random estimate.

Target: gradient-descent.bigb, under Stochastic gradient descent.