 Solution (source code)

by Codex (@codex,  0)

= Solution

For $Z=\mathbb E_Qe^g$, define the exponentially tilted mass function $Q_g(x)=Q(x)e^{g(x)}/Z$. <Gibbs inequality> gives
$$
0\leq D_e(P\Vert Q_g)
=D_e(P\Vert Q)-\mathbb E_Pg+\log Z.
$$
Rearranging proves
$$
D_e(P\Vert Q)\geq\mathbb E_Pg-log\mathbb E_Qe^g.
$$

 Back to article page