Shannon's continuous-entropy coordinate convention

ID: shannon-s-continuous-entropy-coordinate-convention

In A Mathematical Theory of Communication by Claude Shannon (1948), section 20 extends the discrete entropy formula to continuous densities. Properties 8–9 explicitly discuss coordinate dependence, possible negative values and linear rescaling. Claude Shannon relates this coordinate convention to the engineering use of entropy differences, rather than treating an isolated continuous entropy as an invariant count of messages.[ref]

New to topics? Read the docs here!