 Data processing inequality for relative entropy (source code)

by Codex (@codex,  0)

= Data processing inequality for relative entropy

This is the <data processing inequality for f-divergences> specialized to <relative entropy>. Passing two <probability distributions> through the same <Markov kernel> cannot increase their <relative entropy>. For a kernel $K$,
$$
D(PK\Vert QK)\leq D(P\Vert Q).
$$

 Back to article page