A Markov decision process specifies states, admissible actions, action-dependent transition probabilities and rewards. A policy chooses actions from observed information; a time-homogeneous problem uses transition and reward rules independent of the date. A Bellman equation optimizes current reward plus the conditional value of subsequent decisions. For nonnegative rewards, finite-horizon values increase as the horizon grows, while identification of their limit with the infinite-horizon value requires an approximation argument for admissible policies.
Articles by others on the same topic
A Markov Decision Process (MDP) is a mathematical framework used to model decision-making in situations where the outcomes are partly random and partly under the control of a decision maker. MDPs are widely used in fields like operations research, economics, robotics, and artificial intelligence, especially for reinforcement learning problems. An MDP is defined by the following components: 1. **States (S)**: A finite set of states that represent the possible situations in which an agent can find itself.