= Uniform-reference information projection maximizes entropy
{title2=$D(q\Vert u)=\ln m-H_e(q)$}
For a <uniform distribution on a finite set> of $m$ outcomes, direct expansion of <relative entropy> gives
$$
D(q\Vert u)=\sum_i q_i\ln(mq_i)=\ln m-H_e(q),
$$
where $H_e$ is <information entropy> in <nats>. Therefore minimizing <relative entropy> from this particular reference is equivalent to maximizing <information entropy> subject to the same constraint. This statement depends on the uniform reference; an arbitrary reference contributes its own weights.
In the three-outcome example, $H_e(q^*)=\ln3-D(q^*\Vert p)\approx0.901235$ <nats>. The <mean> fixes one aggregate quantity, and <information projection> chooses how to distribute the remaining uncertainty. This extends <maximum entropy on a finite alphabet>, whose unconstrained maximizer is the uniform law itself.
Back to article page
