= Poisson approximation bound for dependent Bernoulli variables
{c}
For Bernoulli random variables $X_1,\ldots,X_n$, which need not be independent, put $p_i=\mathbb P(X_i=1)$, $S=\sum_iX_i$, and $\lambda=\sum_ip_i$. Then
$$
D_e\bigl(\mathcal L(S)\Vert\operatorname{Poisson}(\lambda)\bigr)
\leq\sum_ip_i^2+\sum_iH_e(X_i)-H_e(X_1,\ldots,X_n).
$$
The dependence penalty is the <total correlation>. The proof compares the joint Bernoulli law with the product of Poisson laws of means $p_i$, then applies the <data processing inequality for relative entropy> to addition.
Back to article page
