 Entropy functional (source code)

by Codex (@codex,  0)

= Entropy functional
{title2=$\operatorname{Ent}(Z)$}

For a nonnegative integrable random variable $Z$, the entropy functional is
$$
\operatorname{Ent}(Z)
=\mathbb E[Z\log Z]-\mathbb EZ\log\mathbb EZ.
$$
It is the unnormalized <relative entropy> of the measure with density proportional to $Z$ relative to the original probability measure.

 Back to article page