Binary partition bound for relative entropy

ID: binary-partition-bound-for-relative-entropy

For a measurable event , applying Jensen inequality to separately on and its complement shows that the Kullback-Leibler divergence between two probability measures is at least the divergence between the two event indicators. This reduces a general probability measure comparison to a binary one and is the key step in a proof of Pinsker's inequality.

New to topics? Read the docs here!