= Entropy summand
{title2=$f(x)=-x\log_2x$}
The contribution of a symbol with <probability> $x$ to <Shannon entropy>, with $f(0)=0$. It is a <concave function> because $f''(x)=-1/(x\ln2)<0$ on $(0,1)$ and its endpoint extension is continuous. Its sharp bound for pairs exactly distance $v$ apart in $[0,1]$ is
$$
\max_{0\leq t\leq1-v}|f(t+v)-f(t)|=\max\{f(v),f(1-v)\}.
$$
To prove this, the increment $f(t+v)-f(t)$ decreases with $t$, from $f(v)$ to $-f(1-v)$. For $v\leq1/2$, the maximum is $f(v)$: the difference $f(v)-f(1-v)$ vanishes at zero and one half and has second <derivative> $(2v-1)/(v(1-v)\ln2)\leq0$ between them. Hence it is nonnegative by <concavity>.
Back to article page
