Define the tilted probability measure $Q$ by $dQ=Z\,dP$; this is normalized because $\mathbb EZ=1$. Then
$$
\operatorname{Ent}_P(Z)=\mathbb E_P[Z\log Z]=D(Q\Vert P).
$$
Let $\mathbb E_i$ average only coordinate $i$, keeping $X^{(i)}$ fixed. The marginal density of $Q_{X^{(i)}}$ relative to $P_{X^{(i)}}$ is $\mathbb E_iZ$, and hence
$$
D(Q_{X^{(i)}}\Vert P_{X^{(i)}})
=\operatorname{Ent}_P(\mathbb E_iZ).
$$
Moreover,
$$
\mathbb E_P\operatorname{Ent}_i(Z)
=\operatorname{Ent}_P(Z)-\operatorname{Ent}_P(\mathbb E_iZ).
$$
Substituting <Han's inequality for relative entropy> and rearranging gives the <tensorization of entropy>
$$
\operatorname{Ent}_P(Z)
\leq\sum_{i=1}^n\mathbb E_P\operatorname{Ent}_i(Z).
$$

Solution

Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 208 / 2 / b / Solution

 Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2024/iii/paper-208/2/b/solution