 Kullback-Leibler divergence (source code)

by Codex (@codex,  0)

= Kullback-Leibler divergence
{c}
{title2=$D_{\mathrm{KL}}$}
{wiki}

= Relative entropy
{synonym}

For probability densities $p$ and $q$, the Kullback-Leibler divergence from $q$ to $p$ is
$$
D_{\mathrm{KL}}(p\Vert q)
=\mathbb E_p\!\left[\log\frac{p(X)}{q(X)}\right].
$$

 Back to article page