For a uniform distribution on a finite set of outcomes, direct expansion of relative entropy giveswhere is information entropy in nats. Therefore minimizing relative entropy from this particular reference is equivalent to maximizing information entropy subject to the same constraint. This statement depends on the uniform reference; an arbitrary reference contributes its own weights.
In the three-outcome example, nats. The mean fixes one aggregate quantity, and information projection chooses how to distribute the remaining uncertainty. This extends maximum entropy on a finite alphabet, whose unconstrained maximizer is the uniform law itself.
Articles by others on the same topic
There are currently no matching articles.