EE 276: Readings

Books

• T. M. Cover and J. A. Thomas, Elements of Information Theory.

• M. M. Wilde, From Classical to Quantum Shannon Theory. (Chapter 2 contains a nice summary of classical information theory)

• Soni, Jimmy, and Rob Goodman. A mind at play: How Claude Shannon invented the information age. Simon and Schuster, 2017. [Amazon] (Biography of Claude Shannon)

• MacKay, David JC. Information theory, inference and learning algorithms. Cambridge university press, 2003. [online] (Really cool book on information theory and learning with lots of illustrations and applications)

Papers

• Shannon, Claude E. “Prediction and entropy of printed English.” Bell system technical journal 30.1 (1951): 50-64. [pdf] (Shannon's attempt at estimating the entropy of English by experimenting with human compressors).

• Shannon, Claude E. “A mathematical theory of communication.” Bell system technical journal, 27.3 (1948), 379-423. [pdf] (Shannon's original paper that gave birth to information theory).

Videos

• Compressor head Youtube playlist with fun videos on compression by Google Developers

• The Information Age Youtube playlist by ITSOC and Art of the Problem on concepts in information theory

• The Informaticists Youtube channel: IT-forum and other talks at Stanford

Fun

• The Alice and Bob After Dinner Speech

• Are Pop Lyrics Getting More Repetitive?