Momentum buffer ($b_k$)

A momentum buffer stores an accumulated history of gradients for an optimization algorithm. In one common convention, a parameter vector ${\theta }_k$, supplied gradient $g_k$, learning rate ${\eta }_k$ and constant coefficient $0\le \mu <1$ obey the recurrence relationThe buffer belongs to the optimizer state: it remembers earlier inputs even when the current parameters are unchanged. This is the undampened momentum convention used by PyTorch stochastic gradient descent with weight decay and the Nesterov accelerated gradient method disabled; its documented first buffer is the first gradient, agreeing with this recurrence from $b_0=0$. Other conventions absorb the learning rate into the stored buffer, so a buffer value needs its update convention to be interpreted.^[ref]

Repeated substitution gives the exponentially weighted gradient historyTo verify this identity, substitute its expression for $b_k$ into $b_{k+1}=\mu b_k+g_k$: multiplication by $\mu$ raises every old exponent by one, and the added $g_k$ is precisely the new exponent-zero term. The formula holds at $k=0$ because its sum is empty, completing the mathematical induction.

For an original scalar example with $b_0=0$ and a constant supplied gradient $g_j=1$, the geometric series yields $b_k=(1-{\mu }^k)/(1-\mu )$. At $\mu =0.9$, $b_{10}\approx 6.5132$ and the limit is 10. The buffer is an unnormalized sum: multiplying it by $1-\mu$ changes its scale, and would require a corresponding learning rate adjustment to preserve these parameter updates.

The same memory explains why momentum can temporarily oppose the current gradient. If $b_k=10$, $\mu =0.9$ and the new scalar gradient is $g_k=-1$, then $b_{k+1}=8$. With a positive learning rate, the parameter still decreases, although the present negative gradient alone would call for an increase. Persistence can carry motion through rapidly alternating gradients, but this example also shows why the rule need not decrease the objective function at every step.

A momentum buffer can retain enough earlier history that it points against the current gradient. For the undampened rulethe parameter step is $\eta d_k$ for positive learning rate $\eta$. If $g_k^T{b}_{k+1}<0$, then the directional derivative in direction $d_k$ is $g_k^T{d}_k>0$. The actual update direction is therefore locally uphill for a differentiable objective function, even though gradient descent would use the opposite direction. This history-dependent recurrence is the non-Nesterov momentum convention documented for stochastic gradient descent with dampening and weight decay disabled.^[ref]

For an original scalar example, let $f(\theta )={\theta }^2/2$, current $\theta =1$, stored buffer $b=-3$, coefficient $\mu =1/2$ and $\eta =0.1$. The new gradient is 1, butThe objective increases from $f(1)=0.5$ to $f(1.05)=0.55125$. This is a fully specified finite step, not only a first-order prediction.

More generally,An old buffer sufficiently opposed to the new gradient makes this negative. At $\mu =0$, that mechanism disappears. The example explains why a momentum method need not decrease the objective at every step; it does not imply that retaining gradient history always harms convergence. A descent direction test concerns the direction at the current point, while a convergence claim concerns an entire sequence.

An exponentially weighted gradient history assigns geometrically decreasing coefficients to older supplied gradients. For constant $0\le \mu <1$ and momentum buffer recurrence $b_{k+1}=\mu b_k+g_k$,Each additional update multiplies an old contribution by $\mu$. The formula follows by substituting the expression for $b_k$ into the recurrence and adding the newest exponent-zero term. The same undampened recurrence, including its initialization convention, appears in the primary stochastic gradient descent documentation.^[ref]

For an original example, choose $b_0=0$, $\mu =1/2$ and gradients $g_0=2$, $g_1=-2$, $g_2=4$. Successive buffers are 2, −1, 3.5. Directly expanding the third buffer givesThe newest contribution has coefficient 1; the oldest has coefficient 1/4. The buffer is a weighted sum, not yet a weighted mean. At $b_0=0$, its coefficient sum is the geometric seriesHere $W_3=1.75$, so the normalized weighted mean is $b_3/W_3=2$, rather than the ordinary mean $4/3$.

The coefficients also give a concrete memory scale. In the infinite geometric sequence, the fraction assigned to the newest $m$ gradients is $1-{\mu }^m$. At $\mu =0.9$, 29 gradients account for at least 95% of that coefficient mass, because $0.9^{29}<0.05$ whereas $0.9^{28}>0.05$. This is a statement about coefficients, not a guarantee that gradients at different times are independent or that the optimizer averages a stationary process.