 Hamilton–Jacobi–Bellman equation

For a controlled ordinary differential equation $\dot{x}=f(x,u,t)$, running cost $L(x,u,t)$ and terminal cost $\Phi (x)$, a smooth optimal value function satisfiesThis is the continuous-time Bellman equation. Split the cost into a short initial interval and the optimal continuation. Expanding the continuation by the chain rule, dividing by the interval length and taking its limit yields the equation. The optimization acts on the control at the current state and time.

For $\dot{x}=u$ and running cost $x^2+u^2$, the minimized expression is $x^2+u^2+V_{x}u$. Completing the square gives the minimizing control $u=-V_x/2$ and the equation $V_t+x^2-V_x^2/4=0$. The quadratic ansatz leads to linear-quadratic optimal control. ^[ref]

For a controlled diffusion process $dX=fdt+\sigma dW$, the corresponding equation adds $\frac{1}{2}\mathrm{tr}(\sigma {\sigma }^T{D}_x^{2}V)$ inside the infimum. A value function need not be differentiable; the viscosity solution formulation handles many such cases. A classical verification argument additionally requires the candidate feedback to be admissible and to attain the indicated infimum.

The Hamilton–Jacobi–Bellman (HJB) equation is a fundamental partial differential equation in optimal control theory and dynamic programming. It provides a necessary condition for an optimal control policy for a given dynamic optimization problem. ### Context In many control problems, we aim to find a control strategy that minimizes (or maximizes) a cost function over time.