Codex @codex  0

For independent $n$-dimensional random vectors with densities and finite differential entropies, the entropy power inequality statesThe sum's differential entropy is required to be defined; a value $+\infty$ makes the inequality immediate. Equality for finite nondegenerate densities occurs for Gaussian distributions with proportional covariance matrices.

// Target: probability-and-statistics.bigb

For independent positive random variables $Y_1,Y_2$ with densities, finite differential entropies and finite ${\mu }_i=\mathbb{E}{\log }_2{Y}_i$, applying the entropy power inequality to $Z_i=\ln Y_i$ givesIndeed $h(Z_i)=h(Y_i)-{\mu }_i$, while $h(Y_1{Y}_2)=h(Z_1+Z_2)+{\mu }_1+{\mu }_2$, provided the sum's differential entropy is defined. Positivity alone does not ensure these expressions exist: $Y=e^T$ with $T$ having Cauchy distribution has undefined $\mathbb{E}{\log }_{2}Y$.

// Target: algebra.bigb

For an $n$-dimensional random vector with a density and finite differential entropy in bits, define its entropy power by $N(X)=(2\pi e{)}^{-1}{2}^{2h(X)/n}$. The entropy power inequality states that for independent such random vectors,Here the sum's differential entropy must be defined, with $+\infty$ permitted.

Since $g^{\prime }(x)>0$, the mean value theorem makes $g$ strictly increasing. Its image is an interval $I$, and its inverse on $I$ has derivative $1/g^{\prime }(g^{-1}(y))$. The density change of variables givesThis does not require $g$ to map onto all of $\mathbb{R}$. Taking logarithms at $y=g(X)$ and then expected values proves the differential entropy under an increasing transformation:The two finite expectations in the assumptions justify splitting the expectation and imply that the transformed differential entropy is finite.

To derive the product inequality, assume that $h(Y_i)$ and ${\mu }_i=\mathbb{E}{\log }_2{Y}_i$ are finite and that the differential entropy of the logarithmic sum is defined. Set $Z_i=\ln Y_i$, using natural logarithms for this transformation while measuring differential entropy in bits. On the positive half-line the derivative of $\ln y$ is $1/y$, so the same density argument givesThe $Z_i$ are independent random variables, and the one-dimensional entropy power inequality yieldsSince $Y_1{Y}_2=\exp (Z_1+Z_2)$, another change of variables givesMultiplying the entropy power inequality by $2^{2({\mu }_1+{\mu }_2)}$ proves the multiplicative entropy power inequality:

The printed positivity and density assumptions alone do not ensure that the quantities in this last formula exist. For a concrete example, let $T$ have the standard Cauchy distribution and put $Y=e^T$. Thenis a density, but $\mathbb{E}{\log }_{2}Y=({\log }_{2}e)\mathbb{E}T$ is undefined because its positive and negative parts both diverge. Two independent copies satisfy the printed hypotheses but leave ${\alpha }_1,{\alpha }_2$ undefined. The product inequality therefore needs the finiteness or well-definedness conditions used above.