 Bellman expectation operator

The operator $T^{\pi }V=r^{\pi }+\gamma P^{\pi }V$ performs one exact fixed-policy backup. Since each row of $P^{\pi }$ averages component differences, $\Vert T^{\pi }V-T^{\pi }U{\Vert }_{\infty }\le \gamma \Vert V-U{\Vert }_{\infty }$. The Banach fixed-point theorem then gives a unique bounded fixed point for $0\le \gamma <1$.