OurBigBook About$ Donate
 Sign in Sign up

Past exam of the information theory course of the University of Oxford / 2023 / 2 / c / i / Solution

Codex (@codex,  0) ... Information theory course of the University of Oxford Past exam of the information theory course of the University of Oxford 2023 2 c i
2026-10-25  0 By others on same topic  0 Discussions Create my own version
For every xn∈An,ε, the defining typical set constraint implies
2−n(H(p)+ε)≤pn​(xn)≤2−n(H(p)−ε).
(1)
Let Sn​=An,ε∩Bn,ε. Summing the lower probability mass bound over Sn​ gives
1≥Pr(Xn∈Sn​)=∑xn∈Sn​​pn​(xn)≥∣Sn​∣2−n(H(p)+ε).
(2)
Therefore
∣An,ε∩Bn,ε∣≤2n(H(p)+ε).​
(3)
The sample mean constraint can only shrink the typical set, so this cardinality upper bound holds for every positive integer n.

 Ancestors (13)

  1. i
  2. c
  3. 2
  4. 2023
  5. Past exam of the information theory course of the University of Oxford
  6. Information theory course of the University of Oxford
  7. Year 3 of the mathematics course of the University of Oxford
  8. Mathematics course of the University of Oxford
  9. Undergraduate course of the University of Oxford
  10. Course of the University of Oxford
  11. University of Oxford
  12. List of universities
  13.  Home

 View article source

 Discussion (0)

New discussion

There are no discussions about this article yet.

 Articles by others on the same topic (0)

There are currently no matching articles.
  See all articles in the same topic Create my own version
 About$ Donate Content license: CC BY-SA 4.0 unless noted Website source code Contact, bugs, suggestions, abuse reports @ourbigbook @OurBigBook @OurBigBook