323 lines
14 KiB
TeX
323 lines
14 KiB
TeX
\documentclass{article}
|
|
\usepackage[utf8x]{inputenc}
|
|
\usepackage[margin=1in]{geometry} % Adjust margins
|
|
\usepackage{caption}
|
|
\usepackage{wrapfig}
|
|
\usepackage{subcaption}
|
|
\usepackage{parskip} % dont indent after paragraphs, figures
|
|
\usepackage{xcolor}
|
|
%\usepackage{csquotes} % Recommended for biblatex
|
|
\usepackage{tikz}
|
|
\usepackage{pgfplots}
|
|
\usepackage{svg}
|
|
\usepackage{multirow}
|
|
\usetikzlibrary{positioning}
|
|
\usepackage{float}
|
|
\usepackage{amsmath}
|
|
\PassOptionsToPackage{hyphens}{url}
|
|
\usepackage{hyperref} % allows urls to follow line breaks of text
|
|
\usepackage{glossaries}
|
|
\usepackage[style=ieee, backend=biber, maxnames=1, minnames=1]{biblatex}
|
|
\addbibresource{correction.bib}
|
|
|
|
\newcommand{\red}[1]{\textcolor{red}{#1}}
|
|
\newacronym{snr}{SNR}{signal-to-noise ratio}
|
|
\newacronym{crc}{CRC}{Cyclic Redundancy Check}
|
|
|
|
|
|
|
|
|
|
\title{Error Correction Codes}
|
|
\date{\today}
|
|
|
|
\begin{document}
|
|
\maketitle
|
|
\section{Introduction}
|
|
When messages are transmitted over real media, errors are inevitably introduced.
|
|
Examples range from mundane sources of errors like scratches on CDs and capacitive coupling of conductors
|
|
to geomagnetic storms.
|
|
Though the source of errors are manifold, they are ubiquitous and need to be accounted for if we want any
|
|
real chance of reading a correct message at the end.
|
|
On a basic level, we can understand that a channel with a lower \textit{\acrfull{snr}},
|
|
i.e. more errors per intended information will require more effort to retain the message information.
|
|
\[ \mathrm{SNR} = \frac{P_{signal}}{P_{noise}} \]
|
|
In an analog system, one might attempt to simply increase transmission power $P_{signal}$
|
|
as is done for example in professional audio equipment.
|
|
This would however require the modification of the transmission channel itself, which is only feasible to a certain degree.
|
|
|
|
In a digital system, we could understand \acrshort{snr} as the probability of a bit being flipped in transit.
|
|
For example, if a binary message is sent electronically, with a 1 being represented by a voltage of one volt
|
|
and a 0 being represented by zero volts,
|
|
the receiver is likely to observe some intermediate value such as .82 volts.
|
|
Using a nearest-neighbor criterion, the receiver might consider anything over .5 volts to be the binary signal 1
|
|
and anything less than .5 volts to be the binary signal 0.
|
|
If the noise level is small on average, most transmissions will be interpreted correctly.
|
|
However, there generally remains a chance that an intended 1 will be received as .47 volts and hence interpreted incorrectly as a 0.
|
|
\cite{enwiki:signal-to-noise}
|
|
|
|
|
|
In this digital system, we can improve communication reliability by using a coding scheme that is tolerant of errors.
|
|
As a naive approach, we will simply transmit our message multiple times.
|
|
This is called a \textit{repetition code}, where the receiver will perform a majority vote over the received bits,
|
|
resulting in error detection for 2 bits used, and error correction for 3+ bits used.
|
|
|
|
\begin{figure}[H]
|
|
\begin{minipage}{.5\textwidth}
|
|
\begin{tabular}{c|ccc}
|
|
received & 0 & \textcolor{red}{1} & 2 \\
|
|
\hline
|
|
read & 0 & ERR & 1 \\
|
|
\end{tabular}
|
|
|
|
\begin{tabular}{c|cccccccc}
|
|
received & 0 & \textcolor{red}{1} & \textcolor{red}{2} & 3 \\
|
|
\hline
|
|
read & 0 & 0 & 1 & 1 \\
|
|
\end{tabular}
|
|
|
|
\begin{tabular}{c|cccccccc}
|
|
received & 0 & \textcolor{red}{1} & \textcolor{red}{2} & \textcolor{red}{3} & 4 \\
|
|
\hline
|
|
read & 0 & 0 & ERR & 1 & 1 \\
|
|
\end{tabular}
|
|
\caption{Error detection and detection for 2,3 and 4 bits used per symbol in a repetition code}
|
|
\label{tab:detection-correction}
|
|
\end{minipage}
|
|
\begin{minipage}{.5\textwidth}
|
|
\begin{tikzpicture}
|
|
\begin{axis}[
|
|
domain=0:1,
|
|
samples=100,
|
|
axis lines=middle,
|
|
xlabel={$p$},
|
|
ylabel={Channel Capacity $C$},
|
|
xmin=0, xmax=1,
|
|
ymin=0, ymax=1.1,
|
|
xtick={0,0.25,0.5,0.75,1},
|
|
grid=both,
|
|
width=8cm,
|
|
height=6cm,
|
|
every axis x label/.style={at={(current axis.right of origin)}, anchor=west},
|
|
every axis y label/.style={at={(current axis.above origin)}, anchor=south},
|
|
]
|
|
\addplot[thick, blue] {1-(-x * log2(x) - (1-x) * log2(1-x))};
|
|
\end{axis}
|
|
\end{tikzpicture}
|
|
\caption{Capacity of a binary channel with crossover probability $p$}
|
|
\label{fig:graph-errorrate}
|
|
\end{minipage}
|
|
\end{figure}
|
|
|
|
Formally, a code can be understood as a function $C : \mathcal{X} \rightarrow \Sigma^*$ mapping data $\mathcal{X}$ to a string
|
|
(code word) over an alphabet set $\Sigma$.
|
|
With a code word $C(x)$, we can relate the length of data word to code word, resulting in the measure of information rate:
|
|
\[R = \frac{\log_q(|C|)}{n}\]
|
|
Given the size $q= |\Sigma|$ of an alphabet, size of a code $|C|$ and length of the code words $n$,
|
|
it relates the information to bits transmitted.
|
|
Because any error correction requires redundancy, i.e. adding bits to pure information, the \textit{information rate} has to decrease
|
|
with increasing error correction.
|
|
As a result, $R \leq 1$ ($<$ if error correction is used) and the redundancy is $1-R$.
|
|
|
|
\subsection{Mathematical Bounds}
|
|
In general, the amount of errors a code can detect or correct
|
|
is determined by the Hamming distance $h$ defined as the number of positions in which neighboring strings (code words) differ.
|
|
|
|
"Karolin" and "Kerstin" differ in 3 letters and thus have a Hamming distance of $h=3$, just as the binary example.
|
|
Given a Hamming distance of $h$, $(h-1)$ errors can be detected and to correct $r$ errors a minimum distance of $h\geq 2r+1$ is required.
|
|
This is analogous to the repetition code already shown in \autoref{tab:detection-correction},
|
|
as the length of our repetition code is directly equivalent to the hamming distance.
|
|
|
|
The Hamming distance of a code is then defined as the minimum pairwise distance between any two code words.
|
|
From it, minimal detection and correction capabilities follow according to the above rules.
|
|
\cite{enwiki:hamming-distance}
|
|
|
|
Given a $q$-ary code with length $n$ and minimum Hamming distance $d$, the Gilbert-Varshamov bound provides a bound for the size of
|
|
the resulting code, i.e. the number of available code words given the conditions.
|
|
Let $\mathcal{A}_q(n,d)$ be the maximum possible size of said code, then the Gilbert-Varshamov bound consists of
|
|
$q^n$ as the number of total possible code words in a $q$-ary code of length n, divided by
|
|
$\sum_{j=0}^{d-1} \binom{n}{j} (q-1)^j$, the size of a ball created around a code word by the required Hamming distance.
|
|
\cite{enwiki:gilbert-varshamov}
|
|
The ball is the geometric interpretation of the space created around each used code word by requiring a minimal hamming distance,
|
|
i.e. requiring that no code word lie closer within the vector space.
|
|
|
|
% TODO kraft mcmillan
|
|
|
|
\begin{figure}[h]
|
|
\begin{minipage}{.3\textwidth}
|
|
\begin{tabular}{c|c}
|
|
karolin & 0111 \\
|
|
k\red{e}r\red{st}in & 0\red{000}
|
|
\end{tabular}
|
|
\end{minipage}
|
|
\begin{minipage}{.3\textwidth}
|
|
\[
|
|
\mathcal{A}_q(n,d) \geq \frac{q^n}{\sum_{j=0}^{d-1} \binom{n}{j} (q-1)^j}
|
|
\]
|
|
\end{minipage}
|
|
\end{figure}
|
|
|
|
|
|
\section{Block Codes}
|
|
Block codes are \textit{memoryless}, meaning that each block is encoded independently using a static dictionary.
|
|
Importantly, they can be described by a tuple $(n,k,d)$ taking in $k$ bits of information outputting $n$ bits of code word
|
|
while maintaining minimum distance $d$,
|
|
implying that the code words produced are of fixed length.
|
|
|
|
|
|
\subsection{Cyclic Redundancy Check}
|
|
A \acrfull{crc} is a method of detecting (correcting) errors by interpreting the information to be sent as a polynomial.
|
|
They are particularly suited for the type of burst error common in storage media such as DVDs.
|
|
Typically an $n$-bit \acrshort{crc} can detect any error burst of length $n$,
|
|
with a chance of also detecting longer error bursts of approximately $1-2^{-n}$.
|
|
|
|
Specification of a \acrshort{crc} code requires definition of a so-called \textit{generator polynomial}.
|
|
It is used as the divisor in a polynomial division taking the message as the dividend.
|
|
The remainder of $n$ bits is then appended to the message, thus requiring a generator polynomial of degree $n$ for the calculation.
|
|
Because the division is calculated in a finite (also known as galois) field, so the operation can be performed
|
|
bitwise-parallel, reducing to a simple bitwise \verb|XOR| in the binary case.
|
|
|
|
One of the simplest error-detection systems besides a repetition code, the parity bit, is in fact a 1-bit \acrshort{crc}.
|
|
Using the generator polynomial $g=x+1$ of degree 1 results in the well-known pattern of extending the code words to achieve
|
|
an even number of 1s.
|
|
\cite{enwiki:crc}
|
|
|
|
\begin{figure}[H]
|
|
|
|
\begin{minipage}{0.7\textwidth}
|
|
To calculate the \acrshort{crc}-1, the message $100$ is first extended by $n$ 0s to $1000$.
|
|
It is then divided by the generator polynomial $g=x+1 \equiv (11_2)$, leaving a remainder of $1$.
|
|
Thus, the code word that should be transmitted is $1001$.
|
|
As a result, any 1-bit flip will be detected, allowing the receiver to request retransmission of the message.
|
|
\end{minipage}
|
|
\hspace{1cm}
|
|
\begin{minipage}{0.2\textwidth}
|
|
\begin{verbatim}
|
|
1000 : 11 = 111
|
|
:11
|
|
=010
|
|
:11
|
|
=010
|
|
:11
|
|
=1
|
|
\end{verbatim}
|
|
\end{minipage}
|
|
\end{figure}
|
|
|
|
\subsection{Interleaving}
|
|
Another mitigation for burst errors is interleaving.
|
|
When strong burst errors occur for short intervals in a channel that is otherwise on average low distortion,
|
|
the idea is to spread out code words over time, i.e. interleave them in a pattern known to both transmitter and receiver.
|
|
|
|
\[
|
|
\begin{array}{cccc}
|
|
(x_{11} & x_{12} & x_{13} & x_{14}) \\
|
|
(x_{21} & x_{22} & x_{23} & x_{24}) \\
|
|
(x_{31} & x_{32} & x_{33} & x_{34}) \\
|
|
(x_{41} & x_{42} & x_{43} & x_{44}) \\
|
|
\end{array}
|
|
\rightarrow
|
|
x_{11} x_{21}x_{31}x_{41}x_{12}x_{22}x_{32}x_{42}x_{13}x_{23}x_{33}x_{43}x_{14}x_{24}x_{34}x_{44}
|
|
\]
|
|
In the above example, we interleave 4 code words $x_1$ through $x_4$ of length 4 by writing them row-wise in a table.
|
|
They are then read and transmitted columnwise, cycling bit by bit through each code word, leaving each of them less vulnerable to a short
|
|
burst error.
|
|
The receiver has to keep a buffer of 4 code words to de-interleave the transmission and then apply correction as usual.
|
|
|
|
\subsection{Reed-Solomon}
|
|
% TODO cyclic code
|
|
Similar to \acrshort{crc}, a Reed-Solomon code also interprets the message as a polynomial.
|
|
This polynomial of degree $k-1$ is uniquely identified by any $k$ evaluation points,
|
|
employing the same property as Shamir's secret sharing.
|
|
By transmitting $n>k$ points, a Reed-Solomon code can detect $t=n-k$ errors or locate and correct up to $\lfloor t/2 \rfloor$ errors.
|
|
|
|
Formally, the message will be $(a_1,a_2,...,a_k)$ coefficients of a polynomial
|
|
\[
|
|
f(x) = a_1 + a_2 x + a_3 x^2+...+a_k x^{k-1} = \sum_{i=1}^{k} a_i x^{i-1}
|
|
\]
|
|
\cite{enwiki:reed-solomon}
|
|
|
|
\clearpage
|
|
\section{Convolutional Codes}
|
|
A convolutional code generates its parity symbols by using a sliding input window of a boolean polynomial function.
|
|
A visualisation is provided in \autoref{fig:convolutional}, where bit inputs are processed by a sliding window of size 3,
|
|
processed by 3 different functions resulting in a tuple $(C_1,C_2,C_3)$ being sent over the channel.
|
|
The rate of such a convolutional code is simply determined by the number of input bits to output bits, in this case $\frac{1}{3}$
|
|
as our encoder will output 3 bits for every new input bit being shifted into the window.
|
|
\begin{figure}[H]
|
|
|
|
\begin{minipage}{0.4\textwidth}
|
|
\centering
|
|
\includesvg[height=5cm]{figures/convolutional.drawio.svg}
|
|
\end{minipage}
|
|
\hfill
|
|
\begin{minipage}{0.4\textwidth}
|
|
\begin{tabular}{cc|ccc|c}
|
|
$S_0S_1$ & In & $C_1$ & $C_2$ & $C_3$ & $S_0'S_1'$ \\
|
|
\hline
|
|
\multirow{2}{*}{00} & 0 & 0 & 0 & 0 & 00 \\
|
|
& 1 & 1 & 0 & 1 & 10 \\
|
|
\hline
|
|
\multirow{2}{*}{10} & 0 & 1 & 1 & 0 & 01 \\
|
|
& 1 & 0 & 1 & 0 & 11 \\
|
|
\hline
|
|
\multirow{2}{*}{01} & 0 & 1 & 1 & 0 & 00 \\
|
|
& 1 & 0 & 1 & 0 & 10 \\
|
|
\hline
|
|
\multirow{2}{*}{11} & 0 & 0 & 0 & 1 & 01\\
|
|
& 1 & 1 & 0 & 0 & 11\\
|
|
\end{tabular}
|
|
\end{minipage}
|
|
\caption{Visualisation and transitions of a convolutional encoder}
|
|
\label{fig:convolutional}
|
|
\end{figure}
|
|
Knowing the underlying function allows us to first create a table that later translates to a \textit{trellis},
|
|
a data structure later used in decoding using the Viterbi algorithm.
|
|
|
|
Specifically, we can understand this window of 3 bits to be 2 bits of state $S_0S_1$
|
|
and 1 bit of input, that is the leftmost bit in the window in \autoref{fig:convolutional}.
|
|
|
|
|
|
|
|
\subsection{Viterbi algorithm}
|
|
The viterbi algorithm operates on a \textit{trellis}, that is a variant of a state machine that shows the transitions
|
|
from $S_0S_1$ to $S_0'S_1'$ like in \autoref{fig:trellis}.
|
|
|
|
\begin{figure}[H]
|
|
\begin{tikzpicture}
|
|
% Draw the nodes
|
|
\node (00) at (0,3) [draw,circle] {00};
|
|
\node (10) at (0,2) [draw,circle] {10};
|
|
\node (01) at (0,1) [draw,circle] {01};
|
|
\node (11) at (0,0) [draw,circle] {11};
|
|
|
|
\node (00') at (3,3) [draw,circle] {00};
|
|
\node (10') at (3,2) [draw,circle] {10};
|
|
\node (01') at (3,1) [draw,circle] {01};
|
|
\node (11') at (3,0) [draw,circle] {11};
|
|
|
|
% Draw arrows between nodes
|
|
\draw[->] (00) -- (00') node[midway, above] {0} ;
|
|
\draw[->] (00) -- (10') node[pos=0.2, below] {1} ;
|
|
\draw[->] (10) -- (01') node[pos=0.2, above] {0} ;
|
|
\draw[->] (10) -- (11') node[midway, above] {1} ;
|
|
\draw[->] (01) -- (00') node[midway, above] {0} ;
|
|
\draw[->] (01) -- (10') node[pos=0.2, below] {1} ;
|
|
\draw[->] (11) -- (01') node[pos=0.2, above] {0} ;
|
|
\draw[->] (11) -- (11') node[midway, below] {1} ;
|
|
\end{tikzpicture}
|
|
\caption{Trellis for the viterbi algorithm}
|
|
\label{fig:trellis}
|
|
\end{figure}
|
|
For a given received sequence, we can then duplicate this trellis for each block of (in this example 3) bits
|
|
to decide which bits were most likely sent by minimizing the hamming distance through the graph.
|
|
As a result, we will decode a minimum error input message by retracing the path through the trellis.
|
|
|
|
\section{Conclusion}
|
|
Numerous different forward error correction methods exist that optimise for different scenarios and vary in complexity.
|
|
In practice, they find applications in data storage and transmission, often in combination of each other,
|
|
with the most interesting implementations being used for extreme scenarios such as satellite communication.
|
|
|
|
\clearpage
|
|
\printbibliography
|
|
\end{document}
|