= Solution
Set $N=|\mathcal X|$, $d_i=|p_i-q_i|$ and $d=\sum_i d_i=\|p-q\|_1$. If $d=0$, the two <probability mass functions> agree and the bound is zero. Otherwise every $d_i\leq d\leq1/2$. The half-interval <entropy summand> bound, with $t=\min(p_i,q_i)$ and $v=d_i$, gives
$$
|f(p_i)-f(q_i)|\leq f(d_i).
$$
By the <triangle inequality>, followed by <Jensen's inequality> for the <concave function> $f$,
$$
\begin{aligned}
|H(p)-H(q)|
&\leq\sum_{i=1}^N|f(p_i)-f(q_i)|
\leq\sum_{i=1}^Nf(d_i)\\
&\leq Nf\left(\frac1N\sum_{i=1}^Nd_i\right)
=-d\log\frac dN.
\end{aligned}
$$
This proves the requested <finite-alphabet entropy continuity bound>:
$$
\boxed{|H(p)-H(q)|\leq-\|p-q\|_1\log\left(\frac{\|p-q\|_1}{|\mathcal X|}\right).}
$$
Back to article page
