= Solution
Again put $dQ=Z\,dP$. With $\mathbb E^{[i]}Z=\mathbb E[Z\mid X_i]$, the conditional density of $Q_{X^{(i)}\mid X_i}$ relative to $P_{X^{(i)}}$ is $Z/\mathbb E^{[i]}Z$. Consequently
$$
D(Q_{X^{(i)}\mid X_i}\Vert P_{X^{(i)}}\mid Q_{X_i})
=\mathbb E\left[
Z\log\frac{Z}{\mathbb E^{[i]}Z}\right]
=\mathbb E\operatorname{Ent}_{[i]}(Z).
$$
Part c now gives the alternative tensorization bound
$$
\operatorname{Ent}(Z)
\leq\frac1{n-1}\sum_{i=1}^n
\mathbb E\operatorname{Ent}_{[i]}(Z).
$$
Unlike the bound in part b, each summand averages over all coordinates other than $i$ while holding $X_i$ fixed.
Back to article page
