Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 208 / 2 / b

Table of contents
- Solution b

Solution

Define the tilted probability measure

Q

by

d Q = Z d P

; this is normalized because

E Z = 1

. Then

Ent_{P} (Z) = E_{P} [Z lo g Z] = D (Q ∥ P) .

Let

E_{i}

average only coordinate

i

, keeping

X^{(i)}

fixed. The marginal density of

Q_{X^{(i)}}

relative to

P_{X^{(i)}}

is

E_{i} Z

, and hence

D (Q_{X^{(i)}} ∥ P_{X^{(i)}}) = Ent_{P} (E_{i} Z) .

Moreover,

E_{P} Ent_{i} (Z) = Ent_{P} (Z) - Ent_{P} (E_{i} Z) .

Substituting Han's inequality for relative entropy and rearranging gives the tensorization of entropy

Ent_{P} (Z) \leq \sum_{i = 1}^{n} E_{P} Ent_{i} (Z) .

 Ancestors (10)

 View article source

 Discussion (0)

There are no discussions about this article yet.

 Articles by others on the same topic (0)

There are currently no matching articles.

  See all articles in the same topic Create my own version