Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 11 6 Solution Created 2026-10-03 Updated 2026-10-14
Use discrete Shannon entropy with base-two logarithms. The standard facts needed are the chain rule for information entropy, conditioning reduces entropy, nonnegativity of conditional entropy, and the fact that a uniform distribution on outcomes has Shannon entropy .
Order the coordinates by their indices and put . Applying the chain rule within a coordinate subset and then adding the missing earlier conditioning coordinates givesOn summing over the family, coordinate occurs at least times. Since each is nonnegative,This proves Shearer's inequality without any independence assumption on the coordinates. For countable alphabets, apply the finite-valued assertion to refining finite coordinate partitions; the standard monotone approximation of Shannon entropy gives the same result with extended values. Differential entropy is not the Shannon entropy intended here.
For the graph application, let its bipartition be , with degrees on and on , where . Choose an independent set uniformly from all such sets, and let be its vertex-indicator vector. Then . For put and .
Each vertex of belongs to exactly of these neighbourhoods. Shearer's inequality therefore gives . Conditional on , a vertex of is forced absent if any neighbor was selected, and otherwise is a free fair binary choice. There are no edges within , and the conditional independent-set distribution is uniform on all these free choices. Henceand
There are possible neighborhood patterns. Give the zero pattern weight and every other pattern weight one. For its distribution , concavity of the logarithm, or Jensen's inequality, givesThis establishes the local weighted-entropy estimate rather than just counting neighborhood patterns.
Edge counting gives ; together with , it yields . Exponentiating the Shannon entropy estimate proves the independent-set bound for biregular graphs:The complete bipartite graph has exactly independent sets, since one chooses a subset of one side and subtracts the double-counted empty set. Disjoint unions of such components attain equality. If exactly one degree parameter is zero, edge counting forces the graph to be edgeless and the displayed bound equals , so it remains valid. If , the denominator is undefined; the edgeless graph has exactly independent sets.
Past exam of the mathematics course of the University of Cambridge 2010 iii Paper 12 4 i Solution Created 2026-10-03 Updated 2026-10-14
Let be the number of independent sets of the biregular graph . Choose one such set uniformly and write . All information entropies below use logarithms to base two, soWe will prove .
We state and justify the entropy facts used. For a finite-valued random variable , with . Its conditional entropy is the mean of this quantity over the conditioning variable. Factoring the joint probabilities proves the chain rule for information entropy, . Conditioning reduces entropy: . To see this, condition on and use concavity of to compare the mixture of the conditional distributions given with their mean entropy.
The precise Shearer inequality needed here is: if subsets of the indices of a finite random vector cover every index at least times, then . To prove it, order the indices. The chain rule for information entropy expandsThe inequality is conditioning reduces entropy. Summing over counts each nonnegative conditional term at least times and proves the assertion.
For every , its vertex neighbourhood has vertices. Every lies in exactly of these neighbourhoods. Hence Shearer inequality givesLet , the probability that no neighbour of is selected. Conditional on a particular selected subset of , a vertex with a selected neighbour is forbidden, and all other vertices of are available. Because there are no edges within , every subset of the available vertices is a valid completion. Uniform choice of makes these completions equally likely. The available coordinates are therefore independent fair binary choices, each of entropy one, and the forbidden coordinates are zero. Averaging their number givesBy the chain rule for information entropy,It remains to bound each bracket. Let , a binary vector of length , with distribution on , and put . Assign weight to the all-zero vector and weight to every nonzero vector. ThenThe first inequality is Jensen's inequality for the concave logarithm; the weights sum to . This explicitly handles zero probabilities and does not assume the neighbourhood indicators are independent.
There are right vertices, so combining the bounds and using givesExponentiating proves the required independent-set bound for biregular graphs. The proof uses both degree constraints: supplies the neighbourhood-cover multiplicity and the conditional-completion weight, while determines the number of neighbourhood patterns.