= Total variation distance on a finite set
{title2=$d_{\mathrm{TV}}(P,Q)=\frac12\sum_x|P(x)-Q(x)|$}
Consider the <probability mass functions> $P=(0.4,0.1,0.2,0.3)$ and $Q=(0.1,0.3,0.3,0.3)$ on the <finite set> $S=\{A,B,C,D\}$. Their difference is $(0.3,-0.2,-0.1,0)$. Thus
$$
\boxed{d_{\mathrm{TV}}(P,Q)=\frac12(0.3+0.2+0.1+0)=0.3.}
$$
The <event> $E=\{A\}$ attains this difference: $P(E)=0.4$ and $Q(E)=0.1$. Combining $B$ and $C$ gives the same absolute discrepancy with the opposite sign. Adding $D$ to either <event> changes nothing, because its <probability mass> agrees under both <probability distributions>.
\Image[/total-variation-probability-discrepancy.png]
{title=<Total variation distance> as the largest <event> discrepancy}
{id=image-total-variation-probability-discrepancy}
{description=An illustrative four-outcome example. The positive and negative excesses each total $0.3$; summing both without the factor $1/2$ counts the same redistribution twice.}
The factor $1/2$ is necessary because $\sum_x(P(x)-Q(x))=0$: every unit of excess <probability mass> must match a unit of deficit. For any <event>, including a negative term cannot increase its signed discrepancy, while omitting a positive term cannot increase it either. Therefore $\{x:P(x)>Q(x)\}$ attains the positive maximum. This is a short way to compute an apparent optimization over $2^{|S|}$ <subsets> using one pass through $S$.
For <probability density functions> $p,q$ relative to the same <measure> $\lambda$, the corresponding identity is $d_{\mathrm{TV}}(P,Q)=\tfrac12\int|p-q|\,d\lambda$. Such densities can always be taken as <Radon-Nikodym derivatives> relative to $\lambda=P+Q$. Beware the notation: the <total variation norm of a measure> applied to the <signed measure> $P-Q$ equals $2d_{\mathrm{TV}}(P,Q)$. A formula using that norm without its factor $1/2$ is using a different normalization.
Back to article page
