Maximum entropy on a finite alphabet
ID: maximum-entropy-on-a-finite-alphabet
An information entropy on at most outcomes is at most , with equality for the uniform distribution on all outcomes. For the uniform reference , the Kullback-Leibler divergence is , proving the bound.
New to topics? Read the docs here!