= Solution
Write $w_i=(Aq)_i$. For each row put $S_i=\{j:a_{ij}>0\}$; the positive row entries $\alpha_i=(a_{ij})_{j\in S_i}$ form a <probability distribution> because the row <sum> is one. When $w_i>0$, define the <probability distribution>
$$
r_i(j)=\frac{a_{ij}q_j}{w_i}\qquad(j\in S_i).
$$
Part (a), applied on this row <support>, gives $d(r_i,\alpha_i)\geq0$. Its weighted <sum> expands as
$$
\begin{aligned}
\sum_{i:w_i>0}w_i d(r_i,\alpha_i)
&=\sum_{i,j}a_{ij}q_j\log_2\frac{q_j}{w_i}\\
&=\sum_jq_j\log_2q_j\sum_i a_{ij}-\sum_iw_i\log_2w_i\\
&=H(w)-H(q).
\end{aligned}
$$
Rows with $w_i=0$ contribute zero, since all $a_{ij}q_j$ in such a row vanish. Using the convention that zero-weight logarithmic terms are zero, the result is
$$
\boxed{H(W)-H(V)=\sum_{i:w_i>0}w_i d(r_i,\alpha_i)\geq0.}
$$
Both stochasticity conditions have a role: row sums normalize the comparison distributions, while column sums preserve the total contribution from $q$.
This is <entropy increase under doubly stochastic mixing>.
Back to article page
