Maximum entropy distribution with a prescribed finite mean

ID: maximum-entropy-distribution-with-a-prescribed-finite-mean

For a finite list of real values , maximize information entropy with , , and . Lagrange multipliers give
The constrained second variation is for any nonzero tangent perturbation, proving uniqueness of the entropy maximum whenever a feasible interior solution exists. The mean has derivative , so for nonconstant it decreases strictly with and ranges between the largest and smallest as ranges over the real line. Thus a finite exists uniquely exactly when .
At an extreme mean, strictly positive probabilities are impossible unless all coincide. If all , the mean constraint is redundant when , the maximum is uniform, and is not identifiable. This finite exponential family is an exponential tilting of the uniform distribution.

New to topics? Read the docs here!