= Temporal-difference learning
{title2=$\delta_t=r_{t+1}+\gamma V(s_{t+1})-V(s_t)$}
A value prediction is updated using a reward and a subsequent prediction: $\delta_t=r_{t+1}+\gamma V(s_{t+1})-V(s_t)$. The linear semi-gradient rule is $\Delta w=\alpha\delta_t x(s_t)$ for $V_w(s)=w^{\mathsf T}x(s)$; the target is held fixed for this update. Terminal-state value is zero in an episodic task.<Learning to predict by the methods of temporal differences by Richard S. Sutton>[Abstract and sections 2–3; Machine Learning 3, pp. 9–44: successive predictions and temporal credit assignment]{ref}
Back to article page
