\documentclass{article}
\usepackage[utf8x]{inputenc}
\usepackage[margin=1in]{geometry} % Adjust margins
\usepackage{caption}
\usepackage{wrapfig}
\usepackage{subcaption}
\usepackage{parskip} % dont indent after paragraphs, figures
\usepackage{xcolor}
%\usepackage{csquotes} % Recommended for biblatex
\usepackage{tikz}
\usepackage{pgfplots}
\usepackage{svg}
\usepackage{multirow}
\usetikzlibrary{positioning}
\usepackage{float}
\usepackage{amsmath}
\PassOptionsToPackage{hyphens}{url}
\usepackage{hyperref} % allows urls to follow line breaks of text
\usepackage{glossaries}
\usepackage[style=ieee, backend=biber, maxnames=1, minnames=1]{biblatex}
\addbibresource{correction.bib}

\newcommand{\red}[1]{\textcolor{red}{#1}}
\newacronym{snr}{SNR}{signal-to-noise ratio}
\newacronym{crc}{CRC}{Cyclic Redundancy Check}




\title{Error Correction Codes}
\date{\today}

\begin{document}
\maketitle
\section{Introduction}
When messages are transmitted over real media, errors are inevitably introduced.
Examples range from mundane sources of errors like scratches on CDs and capacitive coupling of conductors
to geomagnetic storms.
Though the source of errors are manifold, they are ubiquitous and need to be accounted for if we want any
real chance of reading a correct message at the end.
On a basic level, we can understand that a channel with a lower \textit{\acrfull{snr}},
i.e. more errors per intended information will require more effort to retain the message information.
\[ \mathrm{SNR} = \frac{P_{signal}}{P_{noise}} \]
In an analog system, one might attempt to simply increase transmission power $P_{signal}$
as is done for example in professional audio equipment.
This would however require the modification of the transmission channel itself, which is only feasible to a certain degree.

In a digital system, we could understand \acrshort{snr} as the probability of a bit being flipped in transit.
For example, if a binary message is sent electronically, with a 1 being represented by a voltage of one volt
and a 0 being represented by zero volts,
the receiver is likely to observe some intermediate value such as .82 volts.
Using a nearest-neighbor criterion, the receiver might consider anything over .5 volts to be the binary signal 1
and anything less than .5 volts to be the binary signal 0.
If the noise level is small on average, most transmissions will be interpreted correctly.
However, there generally remains a chance that an intended 1 will be received as .47 volts and hence interpreted incorrectly as a 0.
\cite{enwiki:signal-to-noise}


In this digital system, we can improve communication reliability by using a coding scheme that is tolerant of errors.
As a naive approach, we will simply transmit our message multiple times.
This is called a \textit{repetition code}, where the receiver will perform a majority vote over the received bits,
resulting in error detection for 2 bits used, and error correction for 3+ bits used.

\begin{figure}[H]
\begin{minipage}{.5\textwidth}
    \begin{tabular}{c|ccc}
        received & 0 & \textcolor{red}{1} & 2 \\
        \hline
        read & 0 & ERR &  1 \\
    \end{tabular}

    \begin{tabular}{c|cccccccc}
        received & 0 & \textcolor{red}{1} & \textcolor{red}{2} & 3 \\
        \hline
        read & 0 & 0 & 1 & 1  \\
    \end{tabular}

    \begin{tabular}{c|cccccccc}
        received & 0 & \textcolor{red}{1} & \textcolor{red}{2} & \textcolor{red}{3} & 4 \\
        \hline
        read & 0 & 0 & ERR & 1 & 1 \\
    \end{tabular}
\caption{Error detection and detection for 2,3 and 4 bits used per symbol in a repetition code}
\label{tab:detection-correction}
\end{minipage}
\begin{minipage}{.5\textwidth}
\begin{tikzpicture}
  \begin{axis}[
    domain=0:1,
    samples=100,
    axis lines=middle,
    xlabel={$p$},
    ylabel={Channel Capacity $C$},
    xmin=0, xmax=1,
    ymin=0, ymax=1.1,
    xtick={0,0.25,0.5,0.75,1},
    grid=both,
    width=8cm,
    height=6cm,
    every axis x label/.style={at={(current axis.right of origin)}, anchor=west},
    every axis y label/.style={at={(current axis.above origin)}, anchor=south},
  ]
    \addplot[thick, blue] {1-(-x * log2(x) - (1-x) * log2(1-x))};
  \end{axis}
\end{tikzpicture}
\caption{Capacity of a binary channel with crossover probability $p$}
\label{fig:graph-errorrate}
\end{minipage}
\end{figure}

Formally, a code can be understood as a function $C : \mathcal{X} \rightarrow \Sigma^*$ mapping data $\mathcal{X}$ to a string
(code word) over an alphabet set $\Sigma$.
With a code word $C(x)$, we can relate the length of data word to code word, resulting in the measure of information rate:
\[R = \frac{\log_q(|C|)}{n}\]
Given the size $q= |\Sigma|$ of an alphabet, size of a code $|C|$ and length of the code words $n$,
it relates the information to bits transmitted.
Because any error correction requires redundancy, i.e. adding bits to pure information, the \textit{information rate} has to decrease
with increasing error correction.
As a result, $R \leq 1$ ($<$ if error correction is used) and the redundancy is $1-R$.

\subsection{Mathematical Bounds}
In general, the amount of errors a code can detect or correct
is determined by the Hamming distance $h$ defined as the number of positions in which neighboring strings (code words) differ.

"Karolin" and "Kerstin" differ in 3 letters and thus have a Hamming distance of $h=3$, just as the binary example.
Given a Hamming distance of $h$, $(h-1)$ errors can be detected and to correct $r$ errors a minimum distance of $h\geq 2r+1$ is required.
This is analogous to the repetition code already shown in \autoref{tab:detection-correction},
as the length of our repetition code is directly equivalent to the hamming distance.

The Hamming distance of a code is then defined as the minimum pairwise distance between any two code words.
From it, minimal detection and correction capabilities follow according to the above rules.
\cite{enwiki:hamming-distance}

Given a $q$-ary code with length $n$ and minimum Hamming distance $d$, the Gilbert-Varshamov bound provides a bound for the size of
the resulting code, i.e. the number of available code words given the conditions.
Let $\mathcal{A}_q(n,d)$ be the maximum possible size of said code, then the Gilbert-Varshamov bound consists of
$q^n$ as the number of total possible code words in a $q$-ary code of length n, divided by
$\sum_{j=0}^{d-1} \binom{n}{j} (q-1)^j$, the size of a ball created around a code word by the required Hamming distance.
\cite{enwiki:gilbert-varshamov}
The ball is the geometric interpretation of the space created around each used code word by requiring a minimal hamming distance,
i.e. requiring that no code word lie closer within the vector space.

\begin{figure}[h]
  \begin{minipage}{.3\textwidth}
    \begin{tabular}{c|c}
      karolin & 0111 \\
      k\red{e}r\red{st}in & 0\red{000}
    \end{tabular} 
  \end{minipage}
  \begin{minipage}{.3\textwidth}
    \[
    \mathcal{A}_q(n,d) \geq \frac{q^n}{\sum_{j=0}^{d-1} \binom{n}{j} (q-1)^j}
    \]
  \end{minipage}
\end{figure}


\section{Block Codes}
Block codes are \textit{memoryless}, meaning that each block is encoded independently using a static dictionary.
Importantly, they can be described by a tuple $(n,k,d)$ taking in $k$ bits of information outputting $n$ bits of code word
while maintaining minimum distance $d$,
implying that the code words produced are of fixed length.


\subsection{Cyclic Redundancy Check}
A \acrfull{crc} is a method of detecting (correcting) errors by interpreting the information to be sent as a polynomial.
They are particularly suited for the type of burst error common in storage media such as DVDs.
Typically an $n$-bit \acrshort{crc} can detect any error burst of length $n$,
with a chance of also detecting longer error bursts of approximately $1-2^{-n}$.

Specification of a \acrshort{crc} code requires definition of a so-called \textit{generator polynomial}.
It is used as the divisor in a polynomial division taking the message as the dividend.
The remainder of $n$ bits is then appended to the message, thus requiring a generator polynomial of degree $n$ for the calculation.
Because the division is calculated in a finite (also known as galois) field, so the operation can be performed
bitwise-parallel, reducing to a simple bitwise \verb|XOR| in the binary case.

One of the simplest error-detection systems besides a repetition code, the parity bit, is in fact a 1-bit \acrshort{crc}.
Using the generator polynomial $g=x+1$ of degree 1 results in the well-known pattern of extending the code words to achieve
an even number of 1s.
\cite{enwiki:crc}

\begin{figure}[H]
  
\begin{minipage}{0.7\textwidth}
  To calculate the \acrshort{crc}-1, the message $100$ is first extended by $n$ 0s to $1000$.
  It is then divided by the generator polynomial $g=x+1 \equiv (11_2)$, leaving a remainder of $1$.
  Thus, the code word that should be transmitted is $1001$.
  As a result, any 1-bit flip will be detected, allowing the receiver to request retransmission of the message.
\end{minipage}
\hspace{1cm}
\begin{minipage}{0.2\textwidth}
\begin{verbatim}
  1000 : 11 = 111
 :11
 =010
  :11
  =010
   :11
    =1
\end{verbatim}
\end{minipage}
\end{figure}

\subsection{Interleaving}
Another mitigation for burst errors is interleaving.
When strong burst errors occur for short intervals in a channel that is otherwise on average low distortion,
the idea is to spread out code words over time, i.e. interleave them in a pattern known to both transmitter and receiver.
  
\[
\begin{array}{cccc}
  (x_{11} & x_{12} & x_{13} & x_{14}) \\
  (x_{21} & x_{22} & x_{23} & x_{24}) \\
  (x_{31} & x_{32} & x_{33} & x_{34}) \\
  (x_{41} & x_{42} & x_{43} & x_{44}) \\
\end{array}
\rightarrow
x_{11} x_{21}x_{31}x_{41}x_{12}x_{22}x_{32}x_{42}x_{13}x_{23}x_{33}x_{43}x_{14}x_{24}x_{34}x_{44}
\]
In the above example, we interleave 4 code words $x_1$ through $x_4$ of length 4 by writing them row-wise in a table.
They are then read and transmitted columnwise, cycling bit by bit through each code word, leaving each of them less vulnerable to a short
burst error.
The receiver has to keep a buffer of 4 code words to de-interleave the transmission and then apply correction as usual.

\subsection{Reed-Solomon}
% TODO cyclic code
Similar to \acrshort{crc}, a Reed-Solomon code also interprets the message as a polynomial.
This polynomial of degree $k-1$ is uniquely identified by any $k$ evaluation points,
employing the same property as Shamir's secret sharing.
By transmitting $n>k$ points, a Reed-Solomon code can detect $t=n-k$ errors or locate and correct up to $\lfloor t/2 \rfloor$ errors.

Formally, the message will be $(a_1,a_2,...,a_k)$ coefficients of a polynomial
\[
f(x) = a_1 + a_2 x + a_3 x^2+...+a_k x^{k-1} = \sum_{i=1}^{k} a_i x^{i-1}
\]
\cite{enwiki:reed-solomon}

\clearpage
\section{Convolutional Codes}
A convolutional code generates its parity symbols by using a sliding input window of a boolean polynomial function.
A visualisation is provided in \autoref{fig:convolutional}, where bit inputs are processed by a sliding window of size 3,
processed by 3 different functions resulting in a tuple $(C_1,C_2,C_3)$ being sent over the channel.
The rate of such a convolutional code is simply determined by the number of input bits to output bits, in this case $\frac{1}{3}$
as our encoder will output 3 bits for every new input bit being shifted into the window.
\begin{figure}[H]
  
\begin{minipage}{0.4\textwidth}
    \centering
    \includesvg[height=5cm]{figures/convolutional.drawio.svg}
\end{minipage}
\hfill
\begin{minipage}{0.4\textwidth}
  \begin{tabular}{cc|ccc|c}
    $S_0S_1$ & In & $C_1$ & $C_2$ & $C_3$ & $S_0'S_1'$ \\
    \hline
    \multirow{2}{*}{00} & 0 & 0 & 0 & 0 & 00 \\
                        & 1 & 1 & 0 & 1 & 10 \\
    \hline
    \multirow{2}{*}{10} & 0 & 1 & 1 & 0 & 01 \\
                        & 1 & 0 & 1 & 0 & 11 \\
    \hline
    \multirow{2}{*}{01} & 0 & 1 & 1 & 0 & 00 \\
                        & 1 & 0 & 1 & 0 & 10 \\
    \hline
    \multirow{2}{*}{11} & 0 & 0 & 0 & 1 & 01\\
                        & 1 & 1 & 0 & 0 & 11\\
  \end{tabular}
\end{minipage}
    \caption{Visualisation and transitions of a convolutional encoder}
    \label{fig:convolutional}
\end{figure}
Knowing the underlying function allows us to first create a table that later translates to a \textit{trellis},
a data structure later used in decoding using the Viterbi algorithm.

Specifically, we can understand this window of 3 bits to be 2 bits of state $S_0S_1$
and 1 bit of input, that is the leftmost bit in the window in \autoref{fig:convolutional}.



\subsection{Viterbi algorithm}
The viterbi algorithm operates on a \textit{trellis}, that is a variant of a state machine that shows the transitions
from $S_0S_1$ to $S_0'S_1'$ like in \autoref{fig:trellis}.

\begin{figure}[H]
\begin{tikzpicture}
    % Draw the nodes
    \node (00) at (0,3) [draw,circle] {00};
    \node (10) at (0,2) [draw,circle] {10};
    \node (01) at (0,1) [draw,circle] {01};
    \node (11) at (0,0) [draw,circle] {11};

    \node (00') at (3,3) [draw,circle] {00};
    \node (10') at (3,2) [draw,circle] {10};
    \node (01') at (3,1) [draw,circle] {01};
    \node (11') at (3,0) [draw,circle] {11};
    
    % Draw arrows between nodes
    \draw[->] (00) -- (00') node[midway, above] {0} ;
    \draw[->] (00) -- (10') node[pos=0.2, below] {1} ;
    \draw[->] (10) -- (01') node[pos=0.2, above] {0} ;
    \draw[->] (10) -- (11') node[midway, above] {1} ;
    \draw[->] (01) -- (00') node[midway, above] {0} ;
    \draw[->] (01) -- (10') node[pos=0.2, below] {1} ;
    \draw[->] (11) -- (01') node[pos=0.2, above] {0} ;
    \draw[->] (11) -- (11') node[midway, below] {1} ;
\end{tikzpicture}
\caption{Trellis for the viterbi algorithm}
\label{fig:trellis}
\end{figure}
For a given received sequence, we can then duplicate this trellis for each block of (in this example 3) bits
to decide which bits were most likely sent by minimizing the hamming distance through the graph.
As a result, we will decode a minimum error input message by retracing the path through the trellis.

\section{Conclusion}
Numerous different forward error correction methods exist that optimise for different scenarios and vary in complexity.
In practice, they find applications in data storage and transmission, often in combination of each other,
with the most interesting implementations being used for extreme scenarios such as satellite communication.

\clearpage
\printbibliography
\end{document}
