Eligibility trace ($e_t=\gamma \lambda e_{t-1}+x(s_t)$)

A decaying record of recent activity that marks a state or connection as eligible for modification by a later teaching signal. An accumulating value-learning trace is $e_t=\gamma \lambda e_{t-1}+x(s_t)$, giving $\Delta w=\alpha {\delta }_t{e}_t$. Other traces record action-policy gradients or pre/post synaptic events. The update and episode-boundary convention must be specified; an eligibility trace is not automatically identical to the visual activity trace used for trace learning.