Markov policy

A Markov policy uses only the current state and, in the nonstationary case, the current time. A stationary Markov policy has the form $u_t=u_{\ast }(X_t)$ for one measurable selector $u_{\ast }$. It is a restricted class of control policies, whose sufficiency for an optimization problem requires the appropriate dynamic-programming hypotheses.