= Verification by a nonnegative control supermartingale
For a nonnegative classical solution $V$ of a finite-horizon <Hamilton-Jacobi-Bellman equation> with nonnegative running reward $r$, the process $V(X_t,t)+\int_0^t r(X_s,a_s)ds$ is a local <supermartingale> under every admissible control. The <Itô formula> gives nonpositive drift because the chosen control cannot exceed the supremum in the equation. Stop where the <stochastic integral> becomes a true <martingale>, take expectations and apply the <Fatou lemma>. Nonnegativity prevents loss of a lower bound when removing the stops, giving the verification upper bound. A maximizing control attains the bound when its stopped reward processes are <uniformly integrable>.
Back to article page
