Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2006/iii/paper-30/1/b/solution

Introduce a selector with Bernoulli distribution of parameter , and construct by taking its conditional law given to be , and its conditional law given to be . No prescribed joint law of is needed. Summing over the selector gives the desired mixture probability mass function. The conditional entropy is
Since conditioning cannot increase information entropy,
This proves the concavity of information entropy. The argument also covers infinite information entropies with the convention that a zero mixing weight contributes zero. At or , the mixture is the corresponding original law and equality is immediate.

New to topics? Read the docs here!