Average-reward optimal policy ($g={\mathrm{liminf}}_{n\to \infty }{n}^{-1}\mathbb{E}{\sum }_{j=1}^n{r}_j$)

An average-reward optimal policy maximizes long-run expected reward per time step. In a finite recurrent Markov chain, a stationary policy's gain is its expected one-step reward under the stationary distribution. Initial-state effects are encoded by a bias function rather than by the gain. In a communicating finite control problem, the average-reward Bellman equation certifies a policy that is optimal from every initial state.