\documentclass[10pt]{article}
\usepackage{amsfonts,amsthm,amsmath,amssymb}
\usepackage{array}
\usepackage{epsfig}
\usepackage{fullpage}
\usepackage{amssymb}
\usepackage[colorlinks = false]{hyperref}
\usepackage{tikz}
\newcommand{\1}{\mathbbm{1}}
\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}
\newcommand{\x}{\times}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\F}{\mathbb{F}}
\newcommand{\E}{\mathop{\mathbb{E}}}
\renewcommand{\bar}{\overline}
\renewcommand{\epsilon}{\varepsilon}
\newcommand{\eps}{\varepsilon}
\newcommand{\DTIME}{\textbf{DTIME}}
\renewcommand{\P}{\textbf{P}}
\newcommand{\SPACE}{\textbf{SPACE}}
\renewcommand{\Pr}{\textbf{Pr}}
\begin{document}
\input{preamble.tex}
\newtheorem{example}[theorem]{Example}
\theoremstyle{definition}
\newtheorem{defn}[theorem]{Definition}
\handout{CS 229r Essential Coding Theory}{Apr 15, 2020}{Instructor:
Madhu Sudan}{Scribe: Eric Bornstein}{Lecture 21}
\section{Overview}
In this class we give constructions of locally correctable codes. We start with a code of rate 1/2. Then, we show how to get high rate codes that are locally correctable, which was first shown in \cite{1}. Then, we use this to give locally correctable codes for any rate which are locally correctable almost up to half the distance. This distance reaches the Singleton Bound. This was first shown in \cite{2}.
\section{Previous Best Results}
First, we recall the definition of locally correctable codes.
\begin{defn}[Locally Correctable/Decodable Codes]
A code $C \subset \{f : [n] \to \Sigma\}$ is a $(\ell,\eps)-$LCC (locally correctable code) if there exists a decoder $D$ such that for any $g : [n] \to \Sigma, f \in C$ with $\delta(f,g) \le \eps$, then for each $x$, $\Pr(D^g(x) \not = f(x)) \le 1/3 $ and $D$ makes at most $\ell$ queries to $g$.
\end{defn}
In this class we are interested in $\ell = o(n)$. Last class, we looked at constant $\ell$. In fact, we will construct a $\ell = O(n^c)$ for any $c$.
Before the results of \cite{1}, the best known rates were from Bivariate Reed-Muller codes. Specifically, set $n = q^2, [n] \simeq \F_q^2, d = (1-2\eps)q, C = \{f : \F_q^2 \to \F_q | \deg(f) \le d\}$. Here, were are letting the degree of the polynomials be slightly less than $q$.
As we have seen, $\dim(C) = \binom{d+1}{2} \approx (1-2\eps)^2q^2/2 \approx (1-\eps')n/2$ for some $\eps'$.
The local decoder $D$ is similar to the one discussed last class. On input $x$ and oracle $g$, the local decoder picks a random line $(x + tb)$ through $x$. Then, the decoder takes $g(x + tb)$ for all values of $t$. For any codeword $f$, $f(x+tb)$ is a polynomial of degree at most $d$ in $t$, so if $f$ and $g$ agree on this line, the decoder can find the polynomial using $q - 1$ values on the line. This determines the value of $f(x)$. We have $\ell = q = \sqrt{n}$.
Notice that $R < 1/2$ and as $R$ approaches $1/2$, $\delta(C)$ approaches 0.
We give the full theorem here:
\begin{theorem}[Locally Correctability of RM Codes]
For any $\eps$ and $n = q^2$, there exists a Reed-Muller code with dimension $(1-\eps)n / 2$ with distance at least $\eps$. There also exists a local decoder that can decode up to half the distance using $O(\sqrt{n})$ elements.
\end{theorem}
\begin{exercise}
Show that if we use $t$ variables, that the rate of the code will be at most $\frac{1}{t!}$ and that the value of $\ell$ will be about $n^{1/t}$.
\end{exercise}
\begin{claimproof}
The rate claim is due to the rate of the $t$ variable Reed-Muller code. For local correctability, we need $q$ points on a line and $n = q^t$.
\end{claimproof}
There is no clear way to generalize this to get a better rate than $1/2$.
\section{Multiplicity Codes}
We will use multiplicity codes to get higher rates, which was done in \cite{1}. We first show:
\begin{theorem}[Local Correctability of Bivariate Multiplicity codes]
Bivariate multiplicity codes of length $n$ can locally correct a positive fraction of errors using $O(\sqrt{n})$ coordinates. The rate of these codes can go up to $2/3$.
\end{theorem}
Our starting example will be the Bivariate Multiplicity 2 code. This is similar to a Reed-Muller. In addition to encoding the evaluation of a polynomial at several points, we also include the evaluation of the derivatives of the polynomial. Here, if $f(x,y) = \sum c_{i,j} x^i y^j$, then
$f_x(x,y) = \sum i c_{i,j} x^{i - 1}y^j$ and $f_y(x,y) = \sum j c_{i,j} x^{i }y^{j - 1}$. The derivatives are defined in the same way as over $\R$. However, higher order derivatives are slightly different over $\F_{q^r}$ for $r > 1$.
Formally, the encoding of $f : \F_q^2 \to \F_q$ is $\langle f(a,b), f_x(a,b), f_y(a,b) \rangle_{a,b}$. This means that the alphabet $\Sigma = \F_q^3$ because we have three values. There are $q^2$ possible values for $a$ and $b$. So, our code maps polynomials of degree at most $d$ to $(\F_q^3)^{q^2}$. We still have $\dim_q(C) = \binom{d+1}{2} \approx d^2 / 2$, but now our alphabet is three times as large, so $\dim(C) \approx d^2 / 6$.
So far, the changes have not been useful. The benefit of our change is that we can now make $d = (1-2\eps) 2q$, which is twice as big as before. So, our rate is $\dim(C) / n \approx \frac{d^2/6}{n} \approx \frac{4q^2/ 6}{q^2} = 2/3$, which is better than before.
\subsection{Decoding}
Now we will show how to locally correct. This will prove that the code has a positive relative distance. The idea is similar to that of locally correctable Reed-Muller codes.
To find the value for input $a$ and oracle $g$, we first pick a random line $a + tb$. We define $g_{a,b}(t) = g(a+tb)$. If $f$ is a nearby codeword, $\deg(f) \le d < 2q$. Note, we can compute $g'_{a,b}(a+tb)$ at any point by combining $g_x(a+tb)$ and $g_y(a+tb)$. This gives us $q$ evaluations of the polynomial $g_{a,b}$ and the same number of its polynomial. This is enough to determine the polynomial because the polynomial has degree less than $2q$.
To see this, consider functions $f$ and $g$ that agree on $q$ points and have derivatives that agree on those same points. This means that $f-g$ must have zeroes on those points. Because the derivatives agree, $f-g$ must have zeroes of multiplicity at least 2 on those points. This means that $f-g$ has at least $2q$ zeroes (double counting the zeroes). Because $\deg(f - g) < 2q$, we know that $f - g = 0$ and $f = g$.
This will allow us to find $g_{a,b}(a)$ and $g'_{a,b}(a)$. However, we need to find $g_x(a)$ and $g_y(a)$. We can do this by taking another random line $a+tc$ and finding $g'_{a,c}(a)$. Using a linear transformation, we can use these to calculate $g_x(a)$ and $g_y(a)$.
Note that we expect a constant fraction of the points on the line to be errors. So, we need to be able to find $g$ even though there are some errors on the line. This leads us to the following exercise.
\begin{exercise}
Show how to decode the univariate multiplicity 2 code from a positive fraction of errors. Use abstract decoding. The univariate multiplicity code has messages $f : \F_q \to \F_q$ and codewords $_a$.
\end{exercise}
\begin{claimproof}
We use abstract decoding similarly to the regular Reed-Solomon decoding. First note that our field elements are of the form $(a,a') \in \F_q$ where the first coordinate represents the evaluation of a polynomial and the second coordinate represents the evaluation of the derivative. Now, $(a,a') * (b,b') = (ab, ab' + a'b)$, where the second coordinate is using the product rule. We let $\mathcal{E}$ is the univariate multiplicity 2 code with polynomials of degree at most $2\eps q$ and $\mathcal{W}$ is the same for degree at most $d + 2\eps q$. We know that $\mathcal{E}$ has dimension $2\eps q + 1$ and $\mathcal{W}$ has distance $2q -(d + 2\eps q) = 2\eps q$. Lastly, we see $\mathcal{E} * \mathcal{C} \subset \mathcal{W}$. We then apply abstract decoding.
\end{claimproof}
\subsection{Higher rate from Higher Multiplicity}
We can extend this idea to codes with higher order derivatives. A multiplicity $m$ code encodes the evaluation of the function and all its derivatives up to the $(m - 1)$th derivatives. For decoding, this requires $m$ lines to find the $m-1$st derivatives.
\begin{exercise}
Show that the bivariate multiplicity $m$ code has rate $1 - \frac{1}{1+m}$ and $\ell = O(m\sqrt{n})$
\end{exercise}
\begin{claimproof}
To calculate the rate, we need to know two numbers: the number of derivatives and the max degree $d$. Because $f_{xy} = f_{yx}$, the number of derivatives is the number of pairs of whole numbers $(a,b)$ with $a + b < m$. This is $\binom{m+1}{2}$. The max degree is $mq$. The rate is $\frac{\binom{d+1}{2}}{\binom{m+1}{2}q^2} \approx \frac{m}{m+1} = 1 - \frac{1}{m-1}$.
To calculate locally decode, we use the same strategy of picking a random line. Now, to calculate the $(m-1)$th derivatives, we need $m$ distinct lines.
\end{claimproof}
\subsection{Better Locality from More Variables}
For any $\eps > 0$, we can use $t = 1/\eps$ variables (instead of two). For any constant $m$, this gives us $\ell = O(n^{1/t}) = O(n^\eps)$. If we let $m = 1/\eps^2$, we also get rate approaching 1.
\section{Applying AEL}
Once we have good locally correctable codes for rates approaching 1 and a positive fraction of errors, we can use AEL to get good locally correctable codes for any rate. These codes will achieve the Singleton Bound. This was first shown in \cite{2}.
We start with a message $m$. Then, we apply $E_1$, which is a locally correctable code from the previous section with high rate, dimension $k$, and $\ell = k^{o(1)}$. For each coordinate of $E_1(m)$, we apply $E_2$, which is a code of rate $R$ and distance $1 - R - o(1)$. Note, $E_2$ does not need to be locally correctable. Lastly, we use an $\eps-$sampler $B$ to mix the codewords. See Figure \ref{fig:code} for the construction diagram.
\begin{figure}
\centering
\begin{tikzpicture}
\node at (-1.7, 2.5) {$m \in \F_q^k$};
\draw (-1,0) rectangle (0,5);
\draw[thick,->] (.2,2.5) -- (1.8,2.5) node[anchor=north east] {LCC $E_1$};
\foreach [evaluate={
\yy=int(mod(\y + 2,5));
}]\y in {0,1,2,3,4}
{\draw (2,\y + .2) rectangle (3.5,{\y + .8});
\draw[thick,->] (3.7,\y + .5) -- (4.3,\y + .5) node[anchor=north east] {$E_2$};
\draw (4.5,\y + .2) rectangle (6.5,{\y + .8});
\draw (9,\y + .2) rectangle (11,{\y + .8});
\draw[->] (6.7,\y + .5) .. controls (7.4, \y + .5) and (8.1, \yy + .5) .. (8.8, \yy + .5);
\draw[->] (6.7,\y + .5) -- (8.8, \y + .5);
}
\node at (7.75,0) {$\eps-$sampler B};
\end{tikzpicture}
\caption{Construction of Locally Correctable Code}
\label{fig:code}
\end{figure}
To correct any coordinate in the codeword, we need to know the corresponding value on the left side of the $\eps-$sampler, which is in some $E_2$ codeword. To know this, we need to know what message was encoded. To know this, we can use the local correctibility of $E_1$. This requires knowing $k^{o(1)}$ of the $E_2$ messages. We can get these if we can decode the corresponding $E_2$ codewords (with errors). We can get these from the right side of the $\eps-$sampler.
Now, we will trace the errors to see that this works. If there are $\frac{1 - R - \eps'}{2}$ errors, all but $\eps$ of the $E_2$ codewords will have about $\frac{1 - R - \eps'}{2}$. Because of this, we will be able to correctly decode all but the $\eps$ fraction of codewords. This was all we needed to locally correct any of the coordinates.
\bibliographystyle{alpha}
\bibliography{Lec21bib}
\end{document}