CS 225: Pseudorandomness
Fall 2016
SYLLABUS
Course Content
| Topics
| Format and Goals | Prerequisites
| Grading
| Textbook
| Related
Courses
Course meetings: Tue-Thu 1:00pm-2:29pm, Maxwell-Dworkin 323 (33 Oxford Street).
Lecturer: Salil Vadhan
Office: Maxwell-Dworkin 337
Assistant: Carol Harlow (harlow@seas.harvard.edu)
Shopping week office hours: Wed 8/31 1:30-5:30; Tue 9/1 2:30-3:30 (cs225 only); Fri 9/2 10-11, 2-4; Tue
9/6 2:30-3:30 (cs225 only); Wed 9/7 3-5; Thu 9/8 9-11, 4-5 (sign up on door or by
emailing Carol Harlow)
Regular office hours (starting 9/9): Friday 10-11, Tuesday 2:30-3:30 or by appointment
Teaching Fellow: Yi-Hsiu Chen (yihsiuchen@g.harvard.edu)
Office: Maxwell-Dworkin 138
Office Hours/Section: Wednesday 4:00-5:30 or by request.
E-mail address for submitting homeworks: cs225-hw@seas.harvard.edu
Questions should be posted on Piazza.
Course website: http://www.courses.fas.harvard.edu/4869
Course
Content
Over the past few
decades, randomization has become one of the most pervasive paradigms in
computer science. Its widespread use includes:
- Algorithm Design: For a number of important algorithmic problems (including problems
in algebra, statistical physics, and approximate counting), the only
efficient algorithms known are randomized.
- Cryptography: Randomness
is woven into the very way we define security.
- Combinatorial Constructions: Many useful combinatorial objects, such as error-correcting codes
and expander graphs (see below), can be constructed simply by generating
them at random.
- Interactive Proofs: Randomization, together with interactive communication, can also add dramatic efficiency improvements
and novel properties (such as "zero knowledge") to classical
"written" mathematical proofs.
So randomness appears
to be extremely useful in these settings, but we still do not know to what
extent it is really necessary. Thus, in this course we will ask:
Main Question: Can
we reduce or even eliminate the need for randomness in the above settings?
Why do we want to do
this? First, essentially all of the applications of
randomness assume we have a source of perfect randomness one that gives
"coin tosses" that are completely unbiased and independent of each
other. It is unclear whether physical sources of perfect randomness exist
and are inexpensive to access. Second, randomized constructions of
objects such as error-correcting codes and expander graphs often do not provide
us with efficient algorithms for using them; indeed, even writing down a
description of a randomly selected object can be infeasible. Finally, and
most fundamentally, our understanding of computation would be incomplete
without understanding the power that randomness provides.
In this course, we will address the
Main Question via a powerful paradigm known as pseudorandomness.
This is the theory of efficiently generating objects that "look
random", despite being constructed using little or no randomness.
Specifically, we will study several kinds of "pseudorandom" objects,
such as:
- Pseudorandom Generators: These are procedures which stretch a short "seed" of
truly random bits into a long string of "pseudorandom" bits
which cannot be distinguished from truly random by any efficient
algorithm. They can be used to reduce and even eliminate the
randomness used by any efficient algorithm. They are also a
fundamental tool in cryptography.
- Randomness Extractors: These are procedures which extract almost uniformly distributed
bits from sources of biased and correlated bits. Their original
motivation was to allow us to use randomized algorithms even with
imperfect physical sources of randomness, but they have also turned out to
have a wide variety of other applications.
- Expander Graphs: These are graphs which are sparse but nevertheless highly connected.
They have been used to address many fundamental problems in computer
science, on topics such as network design, complexity theory, coding
theory, cryptography, and computational group theory.
- Error-Correcting Codes: These are methods for encoding messages so that even if many of the
symbols are corrupted, the original message can still be decoded. We
will focus on "list decoding", where there are so many
corruptions that uniquely decoding the original message is impossible, but
it is still possible to produce a short list of possible candidates.
Each of the above
objects has been the center of a large and beautiful body of research, and
until recently these corpora were largely distinct. An exciting recent
development has been the realization that all four of these objects are almost the
same when interpreted appropriately. Their intimate connections will
be a major focus of the course, tying together the variety of constructions and
applications of these objects we will cover.
The course will reach the
cutting-edge of current research in this area, covering some results from
within the last year. At the same time, the concepts we will cover are
general and useful enough that hopefully anyone with an interest in the theory
of computation or combinatorics could find the
material appealing.
Topics
- The Power of Randomness
- Examples of
randomized algorithms
- Complexity classes (BPP, RP, RL,...)
- Basic properties, e.g. error reduction
- Basic Derandomization Techniques
- Enumeration
- Nonuniformity
- Nondeterminism
- Method of conditional probabilities
- Pairwise and k-wise independence
- Expander Graphs
- Measures of expansion
- Probabilistic existence
- Random walks on expanders
- Explicit constructions: the zig-zag product
(2000)
- Reingold’s logspace algorithm
for undirected s-t connectivity (2005)
- List-Decodable
Error-Correcting Codes
- Probabilistic existence
- The Johnson Bound
- Reed-Solomon, Reed-Muller, and Hadamard Codes
- Guruswami-Sudan decoding algorithm
- Parvaresh-Vardy and Guruswami-Rudra codes
(2005)
- List-decoding view of expanders
- Unbalanced expanders from Parvaresh-Vardy codes (2006)
- Randomness Extractors
- Weak random sources, entropy measures,
impossibility of deterministic extraction
- Probabilistic existence
- Simulating BPP with a weak random
source
- The Leftover Hash Lemma
- Relation to expanders
and list-decodable codes.
- Extraction from block sources
- Condensers and extractors from Parvaresh-Vardy codes (2006)
- Pseudorandom Generators
- Blum-Micali-Yao
and Nisan-Wigderson definitions
- Survey of BMY-type pseudorandom
generators from one-way functions
- The Nisan-Wigderson generator (from
average-case hardness)
- Unconditional derandomization
of constant-depth circuits
- Worst-case/average-case connections
from locally list-decodable codes
- Evidence that BPP=P
- Relation to extractors: Trevisan’s extractor
(1999)
- Applications
- Some Possible Other Topics
- Recent breakthroughs on 2-source extractors and explicit Ramsey graphs (2015)
- Pseudorandomness for space-bounded computation
Format and Goals
The main components of the course are as follows:
- Reading and commenting: For every class meeting, we will assign reading for you to do in advance. You will be expected to read and comment on this material prior by midnight before lecture, using the online forum NB. See the guidelines on reading and commenting.
- Class participation: Our
class meetings will be very interactive, with you collectively bringing
out the key concepts, ideas, and intuition, as well as working through
the difficult technical material together (with our guidance, of
course). This will demand more of you than a standard lecture-based
course, but the hope is that you will come away with a much deeper
understanding of the material.
- Problem Sets: There
will be 6-7 problems sets (including a take-home exam) during the
course. Problem sets will typically be assigned on Thursdays and due
back roughly two weeks later on Friday no later than 5pm (unless using
late days).
By the end of the course, I hope that you will all be able to:
- Comfortably work with a variety of "pseudorandom objects"
and related topics (e.g. spectral graph theory, entropy measures, the
probabilistic method, universal hashing) that are interesting in their
own right and useful in many different areas,
- Fluidly translate between and take advantage of very different ways of viewing the same mathematical and computational objects,
- Extract
both the high-level ideas and low-level details when reading a text and
identify interesting questions that are not answered,
- Explain and collaboratively work through an advanced subject with your peers,
- Understand
the state of the art in the theory of pseudorandomness as needed to
engage in research in pseudorandomness and/or apply it to other areas.
Prerequisites
This is an advanced
graduate course, so I will be assuming that you have general "mathematical
maturity" and a good undergraduate background in the theory of
computation. One concrete guideline is that you should have had a minimum
of two other courses in the theory of computation, including at least one
graduate course. If you have particularly strong math background, then
there can be a bit more flexibility with this. But if you
haven't had a prior graduate course in the theory of computation (numbered CS
22x at Harvard), you must come speak to me at office hours before registering
for the class.
In terms of topics, I will be
assuming familiarity with the following. In all cases (especially
complexity theory), the more background you have, the better.
- Complexity Theory: P, NP, NP-completeness, reductions (as in CS 121).
- Randomized Algorithms: Exposure to some randomized algorithms, as in CS 124, 223, or 224.
- Algebra: The
basics of groups, (finite) fields, vector spaces, eigenvectors/eigenvalues. Any of CS 226, Math 122-123, AM 106 should be sufficient.
- Other: Basic
discrete probability, graph theory & combinatorics.
Grading &
Problem Sets
The requirements of
the course:
- Reading and commenting on the reading before lecture.
- Participation in class, section, office hours, and the online fora (NB and Piazza).
- Biweekly problem sets.
- Take-home final exam.
The biweekly
problem sets will typically be due on Friday by 5 PM. Your problem set
solutions must be typed and submitted electronically to cs225-hw@seas.harvard.edu. You are
allowed 12 late days for the semester, of which at most 5 can be used on any
individual problem set. (1 late day = 24 hours exactly).
The problem sets may require a lot of thought, so be sure to start them early. You are encouraged to
discuss the course material and the homework problems with each other in small
groups (2-3 people). Discussion of homework problems may include
brainstorming and verbally walking through possible solutions, but should not
include one person telling the others how to solve the problem. In
addition, each person must write up their solutions independently, and these
write-ups should not be checked against each other or passed around.
Textbook
We will be following
my monograph Pseudorandomness. When you post your comments on the reading, please include any corrections or suggestions for improving it!
You may also find the following references
useful. Most of them should be in the libraries, on reserve.
Related
Courses This Term