Again put $dQ=Z\,dP$. With $\mathbb E^{[i]}Z=\mathbb E[Z\mid X_i]$, the conditional density of $Q_{X^{(i)}\mid X_i}$ relative to $P_{X^{(i)}}$ is $Z/\mathbb E^{[i]}Z$. Consequently
$$
D(Q_{X^{(i)}\mid X_i}\Vert P_{X^{(i)}}\mid Q_{X_i})
=\mathbb E\left[
Z\log\frac{Z}{\mathbb E^{[i]}Z}\right]
=\mathbb E\operatorname{Ent}_{[i]}(Z).
$$
Part c now gives the alternative tensorization bound
$$
\operatorname{Ent}(Z)
\leq\frac1{n-1}\sum_{i=1}^n
\mathbb E\operatorname{Ent}_{[i]}(Z).
$$
Unlike the bound in part b, each summand averages over all coordinates other than $i$ while holding $X_i$ fixed.


Solution

Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 208 / 2 / d / Solution

 Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2024/iii/paper-208/2/d/solution