A value function assigns each dynamic programming state and time the best objective function value attainable over the remaining decisions. It is a score, whereas a control policy specifies the actions producing that score. In the 0-1 knapsack problem above, is the score, and choosing the items of weights and attains it. In sequential mathematical optimization, the Bellman equation computes a value function by comparing immediate reward plus continuation value. Use a supremum or infimum when the best score need not be attained by any admissible control policy.
At a point where is finite, this is an affine plane supporting the epigraph from below: for every . A nonzero coefficient of the vertical coordinate permits normalization to the displayed graph form. For a constrained value function, the slope is exactly an attained multiplier in the Strong Lagrangian property. When the value function is differentiable at an interior point of its finite domain, a supporting slope equals its gradient: apply the supporting inequality along both positive and negative multiples of each coordinate direction and take limits.
Articles by others on the same topic
In the context of reinforcement learning and decision making, a **value function** is a function that estimates the expected return (or future rewards) that an agent can achieve from a given state or state-action pair. It plays a fundamental role in evaluating the optimality of policies, guiding the agent's decisions as it seeks to maximize its cumulative rewards over time.
The function being maximized in a optimization problem.