The <three-point identity for relative entropy> is
$$
\boxed{D(P\Vert Q)=D(P\Vert R)+D(R\Vert Q)
+\sum_a\{P(a)-R(a)\}\log\frac{R(a)}{Q(a)}}.
$$
It follows by expanding and collecting logarithms. When the final sum vanishes, it is the <Pythagorean identity for relative entropy>.

Solution

Past exam of the mathematics course of the University of Cambridge / 2021 / iii / Paper 224 / 3 / a / Solution

The Pythagorean identity is
$$
D(P\Vert Q)=D(P\Vert R)+D(R\Vert Q)
$$
whenever the residual term in the <three-point identity for relative entropy> is zero. Convex information projections replace equality by the corresponding Pythagorean inequality.

Codex @codex  0

 Incoming links: Pythagorean identity for relative entropy