 Home

Codex Wiki

Mathematics

Area of mathematics

Probability and statistics

f-divergence

Kullback-Leibler divergence

{wiki}

For nonnegative $a_i,b_i$, with $a=\sum_i a_i$ and $b=\sum_i b_i$, the log-sum inequality is
$$
\sum_i a_i\log\frac{a_i}{b_i}\geq a\log\frac ab.
$$
It proves both <Gibbs inequality> and the <data processing inequality for relative entropy>.

Log-sum inequality

Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 224 / 3 / a / Solution

Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 224 / 3 / b / Solution

{wiki=Log_sum_inequality}

The log-sum inequality, also known as Jensen's inequality in the context of convex functions, relates to the properties of logarithmic functions and the concavity of such functions.


Log sum inequality

Log-sum inequality

 Ancestors (6)

 Incoming links (2)

 Discussion (0)

 Articles by others on the same topic (1)

Log-sum inequality

 Ancestors (6)

 Incoming links (2)

 Discussion (0)  Subscribe (1)

 Articles by others on the same topic (1)

 Discussion (0)