Information theory basics from Chapter 2 of "Elements of Information Theory" by Thomas M. Cover & Joy A. Thomas.
Current main results:
- Information inequality (
information_inequality
)
Current main definitions:
- Probability mass function (
pmf
) - Entropy (
entropy
), KL-divergence (kld
)