Differential entropy under an increasing transformation

ID: differential-entropy-under-an-increasing-transformation

Let have a probability density function on an open interval, and let be continuously differentiable, strictly increasing there, with . The change of variables formula gives
If the differential entropy and the expected absolute logarithm of are finite, taking the expectation of the negative logarithm proves the displayed formula, for a fixed base . Local stretching reduces probability density; its contribution to differential entropy is the expected logarithmic stretch. The map is invertible, so this change does not mean that the transformation destroyed information about .

New to topics? Read the docs here!