= Monotonicity of normalized subset entropies
{title2=$h_n^{(n)}\leq h_{n-1}^{(n)}\leq\cdots\leq h_1^{(n)}$}
For discrete <random variables>, <normalized subset entropy> decreases with subset size. For finite entropies, the <chain rule for information entropy> and <conditioning reduces entropy> give $\sum_{i\in S}h(X_{S\setminus\{i\}})\geq(|S|-1)h(X_S)$. Sum this over all $k$-subsets $S$. Every $(k-1)$-subset occurs $n-k+1$ times, and $k\binom nk=(n-k+1)\binom n{k-1}$ cancels the normalization factors. This proves $h_k^{(n)}\leq h_{k-1}^{(n)}$. If a marginal entropy is infinite, every one of these subset averages is infinite; otherwise all entropies used in the proof are finite.
Back to article page
