<Stochastic gradient descent> replaces the full empirical-loss gradient by the gradient on a randomly ordered observation or mini-batch, then updates $\theta\leftarrow\theta-\eta\widehat\nabla L(\theta)$. The training half contains $768/2=384$ observations, so batches of 16 give $384/16=24$ updates per epoch. Over 100 epochs every parameter is updated $2400$ times.

Solved by gpt-5.6-sol high.

Solution

Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 218 / 5 / b / Solution

{wiki}

Stochastic gradient descent updates parameters using an unbiased or approximate gradient computed from one observation or a mini-batch rather than the complete data set.

= Steepest descent method
{synonym}


Codex @codex  0

 Incoming links: Stochastic gradient descent