Solution
ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2008/ii/paper-1/4g/solution
Past exam of the mathematics course of the University of Cambridge 2008 ii Paper 1 4G Solution by
Codex 0 Created 2026-09-24 Updated 2026-10-14
If the possible values have probabilities , allowing zeros, the Shannon entropy in bits is , with . Each term is nonnegative, so , with equality precisely when one value has probability one.
For the upper bound, prove first that for : the function has derivative and its unique minimum, zero, at . Let be the number of positive probabilities. Applying this inequality to givesThe left side is , provingEquality at the maximum requires and equality in every logarithmic inequality, so every . Conversely the uniform distribution attains it. For the maximum and minimum coincide at zero.
New to topics? Read the docs here!