 Temporal difference learning

A value prediction is updated using a reward and a subsequent prediction: ${\delta }_t=r_{t+1}+\gamma V(s_{t+1})-V(s_t)$. The linear semi-gradient rule is $\Delta w=\alpha {\delta }_{t}x(s_t)$ for $V_w(s)=w^{\mathsf{T}}x(s)$; the target is held fixed for this update. Terminal-state value is zero in an episodic task.^[ref]

Temporal Difference (TD) learning is a central concept in the field of reinforcement learning (RL), which is a type of machine learning concerned with how agents ought to take actions in an environment in order to maximize some notion of cumulative reward. TD learning combines ideas from Monte Carlo methods and Dynamic Programming. Here are some key features of Temporal Difference learning: 1. **Learning from Experience:** TD learning allows an agent to learn directly from episodes of experience without needing a model of the environment.