Dynamic programming principle

An optimal value over a time interval is the supremum of immediate reward plus the conditional optimal continuation value. For a controlled state $X$ and running reward $\ell$, this reads $J(x,t)={\sup }_a\mathbb{E}[{\int }_t^{t+h}\ell (X_s,a_s,s)ds+J(X_{t+h},t+h)\mid X_t=x]$, under the usual admissible concatenation and information conditions. Conditioning and concatenating strategies proves both inequalities: every strategy's continuation is bounded by the value function, while approximately optimal continuations approach that bound. For a smooth diffusion value function, Itô formula and the limit $h\downarrow 0$ yield the Hamilton-Jacobi-Bellman equation.