\documentclass[10pt]{article}
\usepackage{amsfonts,amsthm,amsmath,amssymb}
\usepackage{array}
\usepackage{epsfig}
\usepackage{fullpage}
\newcommand{\inprod}[1]{\left\langle #1 \right\rangle}
\begin{document}
\input{preamble.tex}
\renewcommand{\binset}{}
\handout{CS 229r Essential Coding Theory, Lecture 7}{Feb 14, 2017}{Instructor: Madhu Sudan}{Scribes: Vasileios Nakos}{Lecture 7}
%Hamming Codes, Distance, Examples, Limits, and Algorithms}
\section{Last Lecture}
Last week we talked about Reed-Solomon codes \cite{reed1960polynomial}. The Reed-Solomon codes are $[n,k,n-k+1]_n$ codes that are optimal, in the sense that they match the pigeonhole principle.
On weakness of Reed-Solomon codes is their alphabet size, which has to be at least $n$. A fair amount of work in Coding Theory tries to understand how to fix it this disadvantage. We do not expect to achieve the same rate with smaller alphabet, because of the $q$-ary Plotkin bound which states that $R \leq 1 - \frac{q}{q-1} \delta$. Best known codes achieve $R \geq 1 - \delta - \frac{1}{\sqrt{q}}$.
\section{Today}
In today's lecture, we will see concatenated codes, Reed-Muller codes and BCH codes. All these cods try to reduce the alphabet size while still achieving good rate and small relative distance.
\section{Concatenated Codes}
Today we will talk about concatenated codes \cite{forney1966concatenated}. Take a $(N,K,D)_{Q}$ code, which we will call Outer code ($C_{out}$) and a $(n,k,d)_2$ code which we will call inner code ($C_{in}$). We enforce that $Q = 2^k$. Now this means that $C_{out}$ takes messages of length $K$ in an alphabet of size $2^k$ and encodes them to messages of length $N$, while $C_{in}$ takes messages of length $k$ and encodes them to messages of length $n$ in an alphabet of size $2$. The concatenated code then maps each message $(m_1,\ldots, m_K)$ to $(C_{in}(m_1'), C_{in}(m_2'),\ldots, C_{in}(m_K')$, where $(m_1',m_2',\ldots,m_N') = C_{out}(m_1,\ldots,m_k)$. This means that one can use as an outer code a Reed-Solomon code (which inevitably has a large alphabet) and as an inner code a Hadamard code, which as we will see later has alphabet size $2$, and reduce the alphabet size down to $2$. Such a code is called Justesen code.
It can be proved that the concatenated code is $(N \cdot n, K \cdot k, D \cdot d)_2$. To understand the limits of the technique of code concatenation, a result that is useful is the Zyablov bound, which is a lower bound on the rate $R$ and relative distance of concatenated codes. This bound is the best known bound for the binary code.
There is a very simple construction that matches the Zyablov bound in $\mathrm{poly}(n)$ time, although we will not prove it. We employ the following construction: Let $m_0,\ldots,m_{K-1} \in \mathbb{F}_k$ where $F_K = \{a_1,\ldots, a_K\}$ with $K = 2^k$. Then the message is the polynomial $M(x) = \sum_i m_i x^i$ and the encoding of the message equals $$ when we view each one of these symbols as a $k$ bit sequence. This achieves the Zyablov bound (exercise).
\section{Reed-Muller codes}
In this section we will discuss Reed-Muller codes \cite{muller1954application}. These codes were discovered by Muller and provided a decoding algorithm by Reed. As we mentioned in the previous section, our goal is to reduce the alphabet size.
For univariate codes we need $n \leq q$. For bivariate polynomials we need $n \leq q^2$ and we expect that for multivariate polynomials we need $n \leq q^m$, where $m$ is the degree of the polynomial. So we can start building larger codes with $m \rightarrow \infty$, but what is the price we need to pay ?
The construction of the generalised Reed-Muller codes ($q > 2$) is the following:View messages as $m$-variate polynomials of degree at most $r$ over $\mathbb{F}_q$. The encoding is the evaluations over the whole space, that is $q^m$ points.
Let now $N(m,q,r) = | \{ (r_1,\ldots, r_m) : | 0 \leq r_q \leq q-1, \sum_{i=1}^m r_i\}|$, which is the set of all monomials that can appear in a Reed-Muller code and is a basis of the vector space of all codewords. To understand how $N(m,q,r)$ grows we look at some special cases.
\begin{itemize}
\item A simple case is hen $r