Information Theory in Computer Science (Harvard CS 229r)

General Info:
  • Lecturer: Madhu Sudan;  MD 339; email: first name at cs dot harvard dot edu; Office Hours: Thursday 2:30-4pm
  • TF: TBD
  • Time and Location: TuTh 1-2:30 in MD 119.

Lecture Notes:

  • Lecture 1: Introduction. Shearer's Lemma. Plan for course. (My notes. Scribe notes (tex, pdf). )
  • Lecture 2: Entropy. Compression: Asymptotic and Single-shot. (My notes. Scribe notes (tex, pdf). )
  • Lecture 3: Basics of Entropy, Information, Relative Entropy etc. (My notes. Scribe notes (tex, pdf). )
  • Lecture 4: CANCELLED (Madhu Travelling)
  • Lecture 5: Entropy and Counting - 1: Sums of binomial coefficients. Coin-weighting problem. Presented by Vasileios and Manolis. (Scribe notes (tex, pdf). )
  • Lecture 6: Entropy and Counting - 2: Bregman's lemma. Moore bound. Presented by Vasileios and Manolis. (Scribe notes (tex, pdf).)
  • Lecture 7: Hypothesis testing, total variation distance and Pinsker's lemma. Presented by Angela. (Angela's Notes. Scribe notes (tex, pdf). )
  • Lecture 8: Stability in Shearer's Lemma. Presented by Mustazee. (Scribe notes (tex, pdf).)
  • Lecture 9: Introduction to Communication Complexity. (My notes. Scribe notes (tex, pdf).)
  • Lecture 10: Set Disjointness-I: Information Complexity. Presented by Alex W. (My notes, Alex's notes, Scribed notes (tex, pdf)).
  • Lecture 11: Set Disjointness-II: Hellinger Distance. Presented by Minjae. (My chaotic notes (Part 1, Part 2). Scribed notes (tex, pdf))
  • Lectures 12-14: Direct Sum in Communication Complexity and Internal Information Complexity.
    • Lecture 12: Introduce Internal Information Complexity. Direct Sum for IC. (My notes, Scribe notes (tex, pdf)).
    • Lecture 13: Compression of low IC protocols. (My notes, Scribe notes (tex, pdf))
    • Lecture 14: Information Equals (!) Amortized Communication. Presented by Guannan. (Scribe notes (tex, pdf)).
    • Reading material for above lectures:
      •  Compressing Interactive Communication [BarakBCR], Lecture 16 from Harsha et al. course.
      • Amortized communication complexity. Reading material: [BravermanRao]
  • Lecture 15: Data Structure Lower Bounds via Communication Complexity. (Scribe notes (tex, pdf).)
  • Lecture 16: Streaming lower bounds via Communication Complexity, (Scribe notes (tex, pdf).).
  • Lecture 17: Algorithmic Lovasz Local Lemma. Presented by Themis and Ali. (Scribe notes (tex, pdf).)
  • Lectures 18-19: Parallel Repetition Theorem. Presented by Akshay, Pritish, and Tanay. (Notes for Lecture 18 (tex, pdf). Notes for Lecture 19 (tex, pdf).)
  • Lectures 20-21: Cancelled.
  • Lecture 22: Extension complexity. Presented by Sitan. Reading material: [BravermanMoitra]. Scribe notes (tex, pdf).
  • Lecture 23: Graph Entropy and Sorting. Presented by Jack M. Reading material: Lectures 4 and 5 from Anup Rao's course. Scribe notes (tex, pdf).
  • Lecture 24: Guest lecture by Thomas Steinke on Adaptive Data Analysis. Thomas's notes.
  • Lecture 25: What we didn't cover: Topics chosen by students. See piazza for some posts.

Paper List (still under construction):

"Related" Courses: