 Solution

The same mixing argument with a fixed $q^0\in \mathcal{Q}$ shows thatfor every $q\ge 0$ in $\ker P^T$, including boundary vectors. By the finite-state superhedging alternative, there is $\varphi \in {\mathbb{R}}^d$ such thatwhich is the asserted almost-sure inequality. No strict residual is required here.