Verification by a nonnegative control supermartingale

For a nonnegative classical solution $V$ of a finite-horizon Hamilton-Jacobi-Bellman equation with nonnegative running reward $r$, the process $V(X_t,t)+{\int }_0^{t}r(X_s,a_s)ds$ is a local supermartingale under every admissible control. The Itô formula gives nonpositive drift because the chosen control cannot exceed the supremum in the equation. Stop where the stochastic integral becomes a true martingale, take expectations and apply the Fatou lemma. Nonnegativity prevents loss of a lower bound when removing the stops, giving the verification upper bound. A maximizing control attains the bound when its stopped reward processes are uniformly integrable.