Generalized Kullback–Leibler divergence
ID: generalized-kullback-leibler-divergence
For nonnegative densities of possibly different total masses, this divergence includes the mass correction as well as the logarithmic term. It is the Bregman distance of the maximum-entropy regularization functional at a positive reference . Convexity of proves nonnegativity. For equal masses the correction integrates to zero, recovering the Kullback-Leibler divergence; for probability density functions both masses are one. Use the usual extended-value conventions at zero reference density.
New to topics? Read the docs here!