\documentclass[10pt]{article}
\usepackage{amsfonts,amsthm,amsmath,amssymb}
\usepackage{array}
\usepackage{epsfig}
\usepackage{fullpage}
\usepackage{enumerate}
\begin{document}
\input{preamble.tex}
\renewcommand{\binset}{\bbF_2}
\newcommand{\BF}{\mathbb{F}}
\newcommand{\mcE}{\mathcal{E}}
\newcommand{\mcM}{\mathcal{M}}
\newcommand{\mcW}{\mathcal{W}}
\handout{CS 229r Essential Coding Theory}{Feb 21, 2017}{Instructor: Madhu Sudan}{Scribes: Jeffrey Ling}{Lecture 9}
%Hamming Codes, Distance, Examples, Limits, and Algorithms}
\section{Algorithms in coding theory}
Now that we know about some codes, our goal now is to construct the code, i.e. give the encoding and decoding algorithm.
Given a message, we want to be able to compute encoding $E : \Sigma^k \to \Sigma^n$ efficiently. Conversely, given corrupted message, we want to compute decoding $D : \Sigma^n \to \Sigma^k$ efficiently. Here, efficiently is $\poly(n)$ time\footnote{We assume here $n$ is polynomial in $k$. In general we might want $\poly(k)$ time.}.
\section{Easy problems}
From now on, we only consider linear codes. We then want to build the generator matrix $G$, which is equivalent to building $H$ via polynomial reduction. Any $\poly(n)$ amount of information about $E,D$ is ok. (This is also known as building a polynomial sized circuit).
Note that we allow arbitrary preprocessing time to obtain these (random is ok too, such as in the GV bound).
\subsection{Encoding}
Poly time encoding is easy for linear codes, as the description can be written in poly number of bits. Later we ask if we can do this in time linear in $n$ or $k$.
\subsection{Erasure decoding}
The encoded message in $\Sigma^n$ is corrupted to $(\Sigma \cup ?)^n$. Given the generator matrix, if we remove columns of $G$ corresponding to corruptions, does there exist a message that turns into that decoding? This is solvable as a linear system: possible messages are a linear subspace.
When is this message unique? That is, how many erasures can we allow for? Answer: $d-1$, where $d$ is distance of code.
\subsection{Brute-force decoding algorithms $\Sigma^k \to \Sigma^n$}
\begin{itemize}
\item Enumerate all messages, encode them, and measure distance. Runs in time $\poly(n) |\Sigma^k|$. This is fine for a small number of code words (e.g. Hadamard, dual BCH).
\item Enumerate all possible errors. There are $\binom{n}{t} \cdot |\Sigma^t|$ such if there are $t$ errors we wish to correct.
We can also use the parity check matrix. Recall that with error $e$,
$$(E(m) + e)H = eH$$
since $E(m)H = 0$. We can then build a table of all possible results $eH$, i.e. syndromes.
We can enumerate all errors using a lookup table of syndromes. This converts the big running time to space $\approx \Sigma^{n-k}$, which we can lookup quickly in $poly(n)$.
\end{itemize}
\section{General problem for linear codes}
We define the nearest codeword problem (NCP), and also NCP + preprocessing (NCPP).
\begin{definition}
Nearest codeword problem (NCP).
Input: $G \in \BF_q^{k \times n}$ the generator matrix, $r \in \BF_q^n$ the received message, $t$ bound on number of errors.
Output: message $m$ s.t. $\Delta(mG, r) \leq t$ if such $m$ exists. Else, ``too many errors''.
\end{definition}
We also include an additional ``Promise'' about $G$.
\begin{definition}
Promise: $G$ generates code of distance $> 2t$.
\end{definition}
We are mostly interested in the general case where the Promise is satisfied and Preprocessing allowed. We don't know how to solve this, however! It may be polynomially solvable, but would be surprising to Madhu.
\begin{center}
\begin{tabular}{r|cc}
& Preproc & No preproc \\
\hline
Promise & ? & ? \\
No promise & NP-hard & NP-complete
\end{tabular}
\end{center}
\section{General RS-decoding}
We now consider Reed-Solomon codes. It turns out the general problem is NP-hard.
\begin{definition} NP-hard version of RS-decoding.
Input: $\BF_q, n, k$ the code parameters, $x_1, \ldots, x_n \in \BF_q$ distinct points that we evaluate the message polynomials on, $y_1, \ldots, y_n \in \BF_q$ the received response, $t = n-k-2$ the number of errors.
Decide: if there exists polynomial $p \in \BF_q[x]$ with degree $< k$ s.t.
$$|\{p(x_i) \neq y_i | i \in [n] \}| \leq t$$
\end{definition}
% cleanup
Guruswami + Vardy and GGG give algorithms for $t = n-k-c$ for some constants $c$. However, they both require $q$ to be exponentially large in $n$.
If we set $t = (n-k)/2$, we get the easy version of RS-decoding for which there exists a polynomial time algorithm. Literature on this includes: Peterson '60, Berlekamp-Massey 70s (the most cited ``source'' for decoding BCH codes in the 80s and 90s), Welch-Berlekamp '86 and Gemmell-Sudan '92, Pellikaan '88 and Duursma-Kotter '94
NB: many of the early papers assume codes are cyclic, i.e. rotated codewords are also codewords. This is irrelevant to the decoding algorithm.
\section{Easy RS-decoding}
From now on assume $t \leq (n-k)/2$.
We use the key idea of an \emph{error-locating polynomial}.
\begin{definition} Fix a received vector $y$, and suppose $p$ was the message polynomial. Then define an error-locating polynomial $E$ as one that satisfies
$$E(x_i) = 0 \text{ if } p(x_i) \neq y_i, \quad \deg(E) \leq t$$
\end{definition}
Note that by the definition, $E$ satisfies
$$E(x_i) p(x_i) = E(x_i) y_i$$
Let
$$W(x_i) = E(x_i) y_i \implies y_i = W(x_i) / E(x_i)$$
so that $y_i$ can be expressed as a rational function of $x_i$, with numerator degree $