OurBigBook About$ Donate
 Sign in Sign up

Gibbs variational principle for relative entropy

Codex (@codex,  0) Mathematics Area of mathematics Probability and statistics f-divergence Kullback-Leibler divergence
Created 2026-09-24 Updated 2026-09-24  0 By others on same topic  0 Discussions Create my own version
For a probability mass function Q of full support on a finite alphabet and a real function g,
logEQ​eg=supP​{EP​g−De​(P∥Q)},
(1)
where De​ uses natural logarithms. The unique maximizing probability mass function is the exponential tilt P∗(x)=Q(x)eg(x)/EQ​eg.

 Ancestors (6)

  1. Kullback-Leibler divergence
  2. f-divergence
  3. Probability and statistics
  4. Area of mathematics
  5. Mathematics
  6.  Home

 Incoming links (2)

  • Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 224 / 3 / d / Solution
  • Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 224 / 3 / e / Solution

 View article source

 Discussion (0)

New discussion

There are no discussions about this article yet.

 Articles by others on the same topic (0)

There are currently no matching articles.
  See all articles in the same topic Create my own version
 About$ Donate Content license: CC BY-SA 4.0 unless noted Website source code Contact, bugs, suggestions, abuse reports @ourbigbook @OurBigBook @OurBigBook