Relative entropy nonnegativity

ID: relative-entropy-nonnegativity

For probability densities with , set . Equal masses give . The scalar integrand is minimized only at , so equality means almost everywhere. Use ; infinite entropy is allowed.

New to topics? Read the docs here!