Temporal Difference (TD) learning is a central concept in the field of reinforcement learning (RL), which is a type of machine learning concerned with how agents ought to take actions in an environment in order to maximize some notion of cumulative reward. TD learning combines ideas from Monte Carlo methods and Dynamic Programming. Here are some key features of Temporal Difference learning: 1. **Learning from Experience:** TD learning allows an agent to learn directly from episodes of experience without needing a model of the environment.
Articles by others on the same topic
A value prediction is updated using a reward and a subsequent prediction: . The linear semi-gradient rule is for ; the target is held fixed for this update. Terminal-state value is zero in an episodic task.[ref]
Learning to predict by the methods of temporal differences by Richard S. Sutton. Abstract and sections 2–3; Machine Learning 3, pp. 9–44: successive predictions and temporal credit assignment