Information Theory in Computer Science (Harvard CS 229r, Spring 2019)

General Info:
  • Lecturer: Madhu Sudan;  MD 339; email: first name at cs dot harvard dot edu; Office Hours: TuTh 1:15-2:15pm (tentative)
  • TF: Mitali Bafna: MD 138; email first name dot last name at gmail dot com; Office hours and location: TuTh 4:30-5:30 @ LISE 319.
  • Lecture Time and Location: TuTh 12:00-1:15pm; 60 Oxford Street Room 330.

Other links for the course:

Announcements:

  • Problem Set 2 is out (tex, pdf). Due Tuesday Feb 26, 2019 at 8pm. Canvas link.
  • Problem Set 1 (tex, pdf). Solutions (pdf).
  • Sign up for the course on Piazza. This will be principal forum for discussions and announcements. If you do not have a harvard email, send mail to Madhu to get enrolled.
  • Canvas will be the main mechanism for submitting psets and getting them graded. If you are not (yet) on canvas, submit your psets by email till you get added to canvas.
  • Sign up for scribing! Signup sheet here. Template for scribing (preamble, lect01.tex).

Topics (Tentative), Calendar and Handouts:

  • Lecture 01 (Tue. 01/29): Introduction. Shearer's Lemma. My notes. Scribe notes (tex, pdf).
  • Lecture 02 (Thu. 01/31): Entropy & Compression. Notes. Scribe notes (tex, pdf).
  • Lecture 03 (Tue. 02/05): Conditional Entropy, Information, Divergence. Divergence Theorem. Notes. Scribe notes (tex, pdf).
  • Lecture 04 (Thu. 02/07): Compression: Single-shot compression. Universal Compression. Markovian sources. Notes. Scribe notes (v1.tex, v1.pdf, v2.tex, v2.pdf).
  • Lecture 05 (Tue. 02/12): Universal Compression. Lempel-Ziv. Notes. Scribe notes (tex, pdf).
  • Lecture 06 (Thu. 02/14): Channel Coding. Notes. Scribe notes (tex, pdf).
  • Monday 02/18: Presidents' Day Holiday
  • Lecture 07 (Tue. 02/19): Channel Coding (contd.): Converse theorem. Efficiency issues. Linear Coding and Compression. Notes.
  • Lecture 08 (Thu. 02/21): Polar Coding - 2
  • Lecture 09 (Tue. 02/26): Polar Coding - 3
  • Lecture 10 (Thu. 02/28): Communication Complexity - 1
  • Lecture 11 (Tue. 03/05): Communication Complexity - 2
  • Lecture 12 (Thu. 03/07): Communication Complexity of Set Disjointness - 1
  • Lecture 13 (Tue. 03/12): ... Set Disjointness - 2
  • Lecture 14 (Thu. 03/14): Information Complexity
  • Sat. 3/16- Sun 3/24: Spring break
  • Lecture 15 (Tue. 03/26): TBD ... (In remaining lectures we will cover assorted topics such as lower bounds in Data structures, Streaming; applications in differential privacy, complexity theory/zero knowledge, discrete probability (the Lovasz Local Lemma), etc. Suggestions welcome!)
  • Lecture 16 (Thu. 03/28):
  • Lecture 17 (Tue. 04/02):
  • Lecture 18 (Thu. 04/04):
  • Lecture 19 (Tue. 04/09):
  • Lecture 20 (Thu. 04/11): 
  • Lecture 21 (Tue. 04/16):
  • Lecture 22 (Thu. 04/18):
  • Lecture 23 (Tue. 04/23):
  • Lecture 24 (Thu. 04/25): 
  • Lecture 25 (Tue. 04/30):

Reference Materials:

  • The closest reference material may be notes from the last incarnation of this course - here.
  • The classic background text is Elements of Information Theory by Cover and Thomas. While it will not suffice for this course, it might come in handy.