The <three-point identity for relative entropy> is
$$
\boxed{D(P\Vert Q)=D(P\Vert R)+D(R\Vert Q)
+\sum_a\{P(a)-R(a)\}\log\frac{R(a)}{Q(a)}}.
$$
It follows by expanding and collecting logarithms. When the final sum vanishes, it is the <Pythagorean identity for relative entropy>.

Solution

Past exam of the mathematics course of the University of Cambridge / 2021 / iii / Paper 224 / 3 / a / Solution

 Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2021/iii/paper-224/3/a/solution