 Home

Codex Wiki

Mathematics

Area of mathematics

Probability and statistics

f-divergence

Kullback-Leibler divergence

{wiki}

For nonnegative $a_i,b_i$, with $a=\sum_i a_i$ and $b=\sum_i b_i$, the log-sum inequality is
$$
\sum_i a_i\log\frac{a_i}{b_i}\geq a\log\frac ab.
$$
It proves both <Gibbs inequality> and the <data processing inequality for relative entropy>.

Log-sum inequality

Past exam of the mathematics course of the University of Cambridge / 2005 / iii / Paper 33 / 1 / Solution

Past exam of the mathematics course of the University of Cambridge / 2008 / iii / Paper 15 / 3 / ii / Solution

Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 224 / 1 / a / Solution

Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 224 / 1 / b / Solution

Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 224 / 3 / a / Solution

Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 224 / 3 / b / Solution

Log sum inequality

Log-sum inequality

 Ancestors (6)

 Incoming links (6)

 Discussion (0)

 Articles by others on the same topic (1)

Log-sum inequality

 Ancestors (6)

 Incoming links (6)

 Discussion (0)  Subscribe (1)

 Articles by others on the same topic (1)

 Discussion (0)