= Solution
Define the tilted probability measure $Q$ by $dQ=Z\,dP$; this is normalized because $\mathbb EZ=1$. Then
$$
\operatorname{Ent}_P(Z)=\mathbb E_P[Z\log Z]=D(Q\Vert P).
$$
Let $\mathbb E_i$ average only coordinate $i$, keeping $X^{(i)}$ fixed. The marginal density of $Q_{X^{(i)}}$ relative to $P_{X^{(i)}}$ is $\mathbb E_iZ$, and hence
$$
D(Q_{X^{(i)}}\Vert P_{X^{(i)}})
=\operatorname{Ent}_P(\mathbb E_iZ).
$$
Moreover,
$$
\mathbb E_P\operatorname{Ent}_i(Z)
=\operatorname{Ent}_P(Z)-\operatorname{Ent}_P(\mathbb E_iZ).
$$
Substituting <Han's inequality for relative entropy> and rearranging gives the <tensorization of entropy>
$$
\operatorname{Ent}_P(Z)
\leq\sum_{i=1}^n\mathbb E_P\operatorname{Ent}_i(Z).
$$
Back to article page
