Maximum entropy distribution with a prescribed finite mean
ID: maximum-entropy-distribution-with-a-prescribed-finite-mean
For a finite list of real values , maximize information entropy with , , and . Lagrange multipliers giveThe constrained second variation is for any nonzero tangent perturbation, proving uniqueness of the entropy maximum whenever a feasible interior solution exists. The mean has derivative , so for nonconstant it decreases strictly with and ranges between the largest and smallest as ranges over the real line. Thus a finite exists uniquely exactly when .
At an extreme mean, strictly positive probabilities are impossible unless all coincide. If all , the mean constraint is redundant when , the maximum is uniform, and is not identifiable. This finite exponential family is an exponential tilting of the uniform distribution.
New to topics? Read the docs here!