= Bellman expectation equation
{c}
{title2=$V^\pi=r^\pi+\gamma P^\pi V^\pi$}
For a finite <Markov decision process> under fixed policy $\pi$, bounded reward <vector> $r^\pi$ and $0\le\gamma<1$, splitting the discounted return at its first step gives $V^\pi=r^\pi+\gamma P^\pi V^\pi$. A <stochastic matrix> has supremum-norm operator bound one, so $\sum_{k\ge0}(\gamma P^\pi)^k$ converges and $V^\pi=(I-\gamma P^\pi)^{-1}r^\pi$. This is policy evaluation, not optimization over actions.
Back to article page
