\documentclass{article}
\usepackage[utf8x]{inputenc}
\usepackage[margin=1in]{geometry} % Adjust margins
\usepackage{caption}
\usepackage{wrapfig}
\usepackage{subcaption}
\usepackage{parskip} % dont indent after paragraphs, figures
\usepackage{xcolor}
%\usepackage{csquotes} % Recommended for biblatex
\usepackage{tikz}
\usepackage{pgfplots}
\usetikzlibrary{positioning}
\usepackage{float}
\usepackage{amsmath}
\PassOptionsToPackage{hyphens}{url}
\usepackage{hyperref} % allows urls to follow line breaks of text
\usepackage{glossaries}
\usepackage[style=ieee, backend=biber, maxnames=1, minnames=1]{biblatex}
\addbibresource{correction.bib}

\newcommand{\red}[1]{\textcolor{red}{#1}}
\newacronym{snr}{SNR}{signal-to-noise ratio}
\newacronym{crc}{CRC}{Cyclic Redundancy Check}




\title{Error Correction Codes}
\date{\today}

\begin{document}
\maketitle
\section{Introduction}
When messages are transmitted over real media, errors are inevitably introduced.
Examples range from mundane sources of errors like scratches on CDs and capacitive coupling of conductors
to geomagnetic storms.
Though the source of errors are manifold, they are ubiquitous and need to be accounted for if we want any
real chance of reading a correct message at the end.
On a basic level, we can understand that a channel with a lower \textit{\acrfull{snr}},
i.e. more errors per intended information will require more effort to retain the message information.
\[ \mathrm{SNR} = \frac{P_{signal}}{P_{noise}} \]
In an analog system, one might attempt to simply increase transmission power $P_{signal}$
as is done for example in professional audio equipment.
This would however require the modification of the transmission channel itself, which is only feasible to a certain degree.

In a digital system, we could understand \acrshort{snr} as the probability of a bit being flipped in transit.
For example, if a binary message is sent electronically, with a 1 being represented by a voltage of one volt
and a 0 being represented by zero volts,
the receiver is likely to observe some intermediate value such as .82 volts.
Using a nearest-neighbor criterion, the receiver might consider anything over .5 volts to be the binary signal 1
and anything less than .5 volts to be the binary signal 0.
If the noise level is small on average, most transmissions will be interpreted correctly.
However, there generally remains a chance that an intended 1 will be received as .47 volts and hence interpreted incorrectly as a 0.
\cite{enwiki:signal-to-noise}


In this digital system, we can improve communication reliability by using a coding scheme that is tolerant of errors.
As a naive approach, we will simply transmit our message multiple times.
This is called a \textit{repetition code}, where the receiver will perform a majority vote over the received bits,
resulting in error detection for 2 bits used, and error correction for 3+ bits used.

\begin{figure}[H]
\begin{minipage}{.5\textwidth}
    \begin{tabular}{c|ccc}
        received & 0 & \textcolor{red}{1} & 2 \\
        \hline
        read & 0 & ERR &  1 \\
    \end{tabular}

    \begin{tabular}{c|cccccccc}
        received & 0 & \textcolor{red}{1} & \textcolor{red}{2} & 3 \\
        \hline
        read & 0 & 0 & 1 & 1  \\
    \end{tabular}

    \begin{tabular}{c|cccccccc}
        received & 0 & \textcolor{red}{1} & \textcolor{red}{2} & \textcolor{red}{3} & 4 \\
        \hline
        read & 0 & 0 & ERR & 1 & 1 \\
    \end{tabular}
\caption{Error detection and detection for 2,3 and 4 bits used per symbol in a repetition code}
\label{tab:detection-correction}
\end{minipage}
\begin{minipage}{.5\textwidth}
\begin{tikzpicture}
  \begin{axis}[
    domain=0:1,
    samples=100,
    axis lines=middle,
    xlabel={$p$},
    ylabel={Channel Capacity $C$},
    xmin=0, xmax=1,
    ymin=0, ymax=1.1,
    xtick={0,0.25,0.5,0.75,1},
    grid=both,
    width=8cm,
    height=6cm,
    every axis x label/.style={at={(current axis.right of origin)}, anchor=west},
    every axis y label/.style={at={(current axis.above origin)}, anchor=south},
  ]
    \addplot[thick, blue] {1-(-x * log2(x) - (1-x) * log2(1-x))};
  \end{axis}
\end{tikzpicture}
\caption{Capacity of a binary channel with crossover probability $p$}
\label{fig:graph-errorrate}
\end{minipage}
\end{figure}



\subsection{Mathematical Bounds}
In general, the amount of errors a code can detect or correct
is determined by the Hamming distance $h$ defined as the number of positions in which neighboring strings (code words) differ.

"Karolin" and "Kerstin" differ in 3 letters and thus have a Hamming distance of $h=3$, just as the binary example.
In general, a code is said to have a Hamming distance of $h$ if it is the minimal pairwise distance of all codewords.
Given a Hamming distance of $h$, $(h-1)$ errors can be detected, to correct $r$ errors a minimum distance of $h\geq 2r+1$ is required.
This is analogous to the repetition code already shown in \autoref{tab:detection-correction},
as the length of our repetition code is directly equivalent to the hamming distance.

The Hamming distance of a code is then defined as the minimum pairwise distance between any two code words.
From it, minimal detection and correction capabilities follow according to the above rules.
\cite{enwiki:hamming-distance}

Given a $q$-ary code with length $n$ and minimum Hamming distance $d$, the Gilbert-Varshamov bound provides a bound for the size of
the resulting code, i.e. the number of available code words given the conditions.
Let $\mathcal{A}_q(n,d)$ be the maximum possible size of said code, then the Gilbert-Varshamov bound consists of
$q^n$ as the number of total possible code words in a $q$-ary code of length n, divided by
$\sum_{j=0}^{d-1} \binom{n}{j} (q-1)^j$, the size of a ball created around a code word by the required Hamming distance.
\cite{enwiki:gilbert-varshamov}
The ball is the geometric interpretation of the space created around each used code word by requiring a minimal hamming distance,
i.e. requiring that no code word lie closer within the vector space.

\begin{figure}[h]
  \begin{minipage}{.3\textwidth}
    \begin{tabular}{c|c}
      karolin & 0111 \\
      k\red{e}r\red{st}in & 0\red{000}
    \end{tabular} 
  \end{minipage}
  \begin{minipage}{.3\textwidth}
    \[
    \mathcal{A}_q(n,d) \geq \frac{q^n}{\sum_{j=0}^{d-1} \binom{n}{j} (q-1)^j}
    \]
  \end{minipage}
\end{figure}


\section{Block Codes}
% NOTE fact-check
Block codes are \textit{memoryless}, meaning that each block is encoded independently using a static dictionary.

\subsection{Cyclic Redundancy Check}
A \acrfull{crc} is a method of detecting (correcting) errors by interpreting the information to be sent as a polynomial.
They are particularly suited for the type of burst error common in storage media such as DVDs.
Typically an $n$-bit \acrshort{crc} can detect any error burst of length $n$,
with a chance of also detecting longer error bursts of approximately $1-2^{-n}$.

Specification of a \acrshort{crc} code requires definition of a so-called \textit{generator polynomial}.
It is used as the divisor in a polynomial division taking the message as the dividend.
The remainder of $n$ bits is then appended to the message, thus requiring a generator polynomial of degree $n$ for the calculation.
Because the division is calculated in a finite (also known as galois) field, so the operation can be performed
bitwise-parallel, reducing to a simple bitwise \verb|XOR| in the binary case.

One of the simplest error-detection systems besides a repetition code, the parity bit, is in fact a 1-bit \acrshort{crc}.
Using the generator polynomial $g=x+1$ of degree 1 results in the well-known pattern of extending the code words to achieve
an even number of 1s.
\cite{enwiki:crc}

\begin{figure}[H]
  
\begin{minipage}{0.7\textwidth}
  To calculate the \acrshort{crc}-1, the message $100$ is first extended by $n$ 0s to $1000$.
  It is then divided by the generator polynomial $g=x+1 \equiv (11_2)$, leaving a remainder of $1$.
  Thus, the code word that should be transmitted is $1001$.
  As a result, any 1-bit flip will be detected, allowing the receiver to request retransmission of the message.
\end{minipage}
\hspace{1cm}
\begin{minipage}{0.2\textwidth}
\begin{verbatim}
  1000 : 11 = 111
 :11
 =010
  :11
  =010
   :11
    =1
\end{verbatim}
\end{minipage}
\end{figure}

\subsection{Reed-Solomon}
Similar to \acrshort{crc}, a Reed-Solomon code also interprets the message as a polynomial.
This polynomial of degree $k-1$ is uniquely identified by $k$ evaluation points.
By transmitting $n>k$ points, a Reed-Solomon code can detect $t=n-k$ errors or locate and correct up to $\lfloor t/2 \rfloor$ errors.

Formally, the message will be $(a_1,a_2,...,a_k)$ coefficients of a polynomial
\[
f(x) = a_1 + a_2 x + a_3 x^2+...+a_k x^{k-1} = \sum_{i=1}^{k} a_i x^{i-1}
\]
\cite{enwiki:reed-solomon}

\section{Convolutional Codes}

\printbibliography
\end{document}
