Differential entropy under an increasing transformation
ID: differential-entropy-under-an-increasing-transformation
Let have a probability density function on an open interval, and let be continuously differentiable, strictly increasing there, with . The change of variables formula givesIf the differential entropy and the expected absolute logarithm of are finite, taking the expectation of the negative logarithm proves the displayed formula, for a fixed base . Local stretching reduces probability density; its contribution to differential entropy is the expected logarithmic stretch. The map is invertible, so this change does not mean that the transformation destroyed information about .
New to topics? Read the docs here!