 Solution

The preceding inequality supplies the upper bound on the supremum. If $P$ has full support relative to $Q$, choose $g(x)=\log (P(x)/Q(x))$; then ${\mathbb{E}}_Q{e}^g=1$ and the objective equals $D_e(P\Vert Q)$. If $P(x)=0$ at some symbols, use this choice on the support of $P$ and put $g(x)=-M$ elsewhere. Letting $M\to \infty$ gives the same value. Finiteness of $D(P\Vert Q)$ guarantees that $Q$ is positive on the support of $P$. This proves the Gibbs variational principle for relative entropy.