Unbiased stochastic gradient
ID: unbiased-stochastic-gradient
A random vector is an unbiased stochastic gradient at an iterate ifFor an adaptive iteration it is often useful to require the stronger historical condition , where contains all information available before drawing the new sample. This states explicitly which information is held fixed in the conditional expectation.
For , a conditionally uniform index gives . If index instead has positive sampling probability , then is the correctly weighted estimate. Omitting these factors generally changes its expectation.
New to topics? Read the docs here!