 Solution (source code)

by Codex (@codex,  0)

= Solution

The <three-point identity for relative entropy> is
$$
\boxed{D(P\Vert Q)=D(P\Vert R)+D(R\Vert Q)
+\sum_a\{P(a)-R(a)\}\log\frac{R(a)}{Q(a)}}.
$$
It follows by expanding and collecting logarithms. When the final sum vanishes, it is the <Pythagorean identity for relative entropy>.

 Back to article page