 Home

Codex Wiki

Mathematics

Area of mathematics

Probability and statistics

f-divergence

Kullback-Leibler divergence

{c}
{wiki}

For <probability distribution>[probability distributions] $p$ and $q$ on the same finite set,
$$
D_{\mathrm{KL}}(p\Vert q)
=\sum_i p_i\log\frac{p_i}{q_i}\geq0,
$$
with equality exactly when $p=q$ on the support of $p$.

Gibbs inequality

Log-sum inequality

Past exam of the mathematics course of the University of Cambridge / 2004 / iii / Paper 32 / 4 / Solution

Past exam of the mathematics course of the University of Cambridge / 2005 / iii / Paper 33 / 1 / Solution

Past exam of the mathematics course of the University of Cambridge / 2006 / iii / Paper 30 / 1 / a / Solution

Past exam of the mathematics course of the University of Cambridge / 2007 / ii / Paper 1 / 4G / Solution

Past exam of the mathematics course of the University of Cambridge / 2009 / ii / Paper 1 / 12H / iii / Solution

Past exam of the mathematics course of the University of Cambridge / 2009 / ii / Paper 1 / 12H / i / Solution

Past exam of the mathematics course of the University of Cambridge / 2017 / iii / Paper 216 / 4 / a / Solution

Past exam of the mathematics course of the University of Cambridge / 2017 / ii / Paper 2 / 11G / Solution

Past exam of the mathematics course of the University of Cambridge / 2018 / ii / Paper 1 / 3H / Solution

Past exam of the mathematics course of the University of Cambridge / 2021 / ii / Paper 1 / 11K / Solution

Past exam of the mathematics course of the University of Cambridge / 2022 / iii / Paper 224 / 4 / c / Solution

Past exam of the mathematics course of the University of Cambridge / 2022 / iii / Paper 224 / 4 / d / Solution

Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 224 / 3 / c / Solution

Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 224 / 1 / c / Solution

Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 224 / 4 / b / Solution

Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 224 / 4 / a / Solution

Gibbs' inequality

Gibbs inequality

 Ancestors (6)

 Incoming links (17)

 Discussion (0)

 Articles by others on the same topic (1)

Gibbs inequality

 Ancestors (6)

 Incoming links (17)

 Discussion (0)  Subscribe (1)

 Articles by others on the same topic (1)

 Discussion (0)