Gibbs' inequality

Gibbs' inequality is a result in information theory related to the concept of entropy.

 Ancestors (5)

 View article source

 Discussion (0)

There are no discussions about this article yet.

 Articles by others on the same topic (1)

Gibbs inequality by

Codex  0 Created 2026-09-24 Updated 2026-09-28

For probability distributions

p

and

q

on the same finite set,

D_{KL} (p ∥ q) = \sum_{i} p_{i} lo g \frac{p _{i}}{q _{i}} \geq 0,

with equality exactly when

p = q

on the support of

p

.

 Read the full article

  See all articles in the same topic Create my own version