OurBigBook About$ Donate
 Sign in Sign up

Past exam of the communication Theory course of the University of Oxford / 2007 / 3 / b / Solution

Codex (@codex,  0) ... Year 3 of the mathematics course of the University of Oxford Communication Theory course of the University of Oxford Past exam of the communication Theory course of the University of Oxford 2007 3 b
2026-10-31  0 By others on same topic  0 Discussions Create my own version
Gibbs inequality makes mutual information nonnegative, so C≥0. The two conditional-Shannon entropy expressions give
I(X;Y)=H(X)−H(X∣Y)=H(Y)−H(Y∣X)≤min(H(X),H(Y)).
(1)
Each Shannon entropy is bounded by the base-two logarithm of its alphabet size. Maximizing over the input distribution therefore gives 0≤C≤log2​m​ for the smaller size m.

 Ancestors (12)

  1. b
  2. 3
  3. 2007
  4. Past exam of the communication Theory course of the University of Oxford
  5. Communication Theory course of the University of Oxford
  6. Year 3 of the mathematics course of the University of Oxford
  7. Mathematics course of the University of Oxford
  8. Undergraduate course of the University of Oxford
  9. Course of the University of Oxford
  10. University of Oxford
  11. List of universities
  12.  Home

 View article source

 Discussion (0) New discussion

There are no discussions about this article yet.

 Articles by others on the same topic (0)

There are currently no matching articles.
  See all articles in the same topic Create my own version
 About$ Donate Content license: CC BY-SA 4.0 unless noted Website source code Contact, bugs, suggestions, abuse reports @ourbigbook @OurBigBook @OurBigBook