The contribution of a symbol with probability to Shannon entropy, with . It is a concave function because on and its endpoint extension is continuous. Its sharp bound for pairs exactly distance apart in isTo prove this, the increment decreases with , from to . For , the maximum is : the difference vanishes at zero and one half and has second derivative between them. Hence it is nonnegative by concavity.
New to topics? Read the docs here!