Optimization epoch

A unit of work corresponding to one full pass through the components used to measure an iterative method. For coordinate descent in $n$ coordinates, count $n$ coordinate partial derivative evaluations as an optimization epoch: updates on blocks of size $m$ require $n/m$ updates per optimization epoch. For stochastic gradient descent on a finite sum of losses, a common different convention counts one evaluation of every loss component. The convention must be specified. Random sampling with replacement need not visit every component during one such unit of work.

Target: gradient-descent.bigb, under Gradient descent.