= Training epoch
A <training epoch> is conventionally one pass through the <training data> used by an iterative learning procedure. Several parameter updates can occur within that pass. A <mini-batch> supplies a group of examples for an update, while the epoch counts progress through the data collection rather than counting individual updates.<Optimizing Model Parameters in the PyTorch tutorials>[Hyperparameters, number of epochs and batch size; Optimization Loop, outer epoch and training loop]{ref}
Suppose there are $N=10$ training examples. Visit each exactly once per pass, form mini-batches of at most $B=4$ examples and retain the smaller final group. Their sizes are $4,4,2$. With one update for every mini-batch, an epoch has
$$
U=\left\lceil\frac{N}{B}\right\rceil=3
$$
updates. Five epochs therefore make fifteen updates and fifty example visits. Fifty visits do not mean fifty distinct examples: the same ten examples are reused.
Changing the mini-batch size changes the update count. Taking all ten examples together gives one update per epoch under the same rule, while groups of two give five. An epoch count alone consequently does not specify the number of <stochastic gradient descent> steps, the amount of computation or the final fitted model.
The finite-pass assumption matters. Dropping an incomplete final group in this example gives two updates and only eight visits during that iterator traversal. Reordering the examples on later passes can change which examples are omitted. Sampling with replacement can repeat some examples before visiting others, so a chosen number of sampled updates need not be a literal once-per-example pass. Report those conventions rather than assuming that every loop named epoch guarantees full coverage.
The <learning rate schedule> needs a declared clock too: changing the <learning rate> once per epoch differs from changing it once per mini-batch. The optimization tutorial separates the outer loop from the operations that clear gradients, perform <backpropagation> and update parameters inside the training loop.<Optimizing Model Parameters in the PyTorch tutorials>[Optimizer, zero_grad/backward/step; Full Implementation, nested training and epoch loops]{ref} Keeping data passes and update steps distinct makes training runs easier to compare.
Back to article page
