Monotonicity of normalized subset entropies ($h_n^{(n)}\le h_{n-1}^{(n)}\le \cdots \le h_1^{(n)}$)

For discrete random variables, normalized subset entropy decreases with subset size. For finite entropies, the chain rule for information entropy and conditioning reduces entropy give ${\sum }_{i\in S}h(X_{S\setminus \{i\}})\ge (|S|-1)h(X_S)$. Sum this over all $k$-subsets $S$. Every $(k-1)$-subset occurs $n-k+1$ times, and $k\left(\genfrac{}{}{0ex}{}{n}{k}\right)=(n-k+1)\left(\genfrac{}{}{0ex}{}{n}{k-1}\right)$ cancels the normalization factors. This proves $h_k^{(n)}\le h_{k-1}^{(n)}$. If a marginal entropy is infinite, every one of these subset averages is infinite; otherwise all entropies used in the proof are finite.