= Entropy increase under doubly stochastic mixing
{title2=$H(Aq)\geq H(q)$}
Let $A$ be a <doubly stochastic matrix>, let $q$ be a <probability vector>, and put $w=Aq$. In row $i$ with $w_i>0$, define $r_i(j)=a_{ij}q_j/w_i$ on the <support> where $a_{ij}>0$. The positive row entries $\alpha_i=(a_{ij})$ form a <probability vector>. Expanding <relative entropy> and using the column sums gives
$$
H(w)-H(q)=\sum_{i:w_i>0}w_i d(r_i,\alpha_i)\geq0.
$$
Zero-output rows contribute zero because all their $a_{ij}q_j$ vanish. The row sums supply the normalized comparison laws, while the column sums preserve the input <Shannon entropy>[entropy] contribution. <Equality> holds exactly when $q_j=w_i$ for every positive <matrix> entry $a_{ij}$, including the automatic zero values in zero-output rows.
Back to article page
