diff --git a/correction.tex b/correction.tex index e65ff35..8cc74cf 100644 --- a/correction.tex +++ b/correction.tex @@ -9,6 +9,8 @@ %\usepackage{csquotes} % Recommended for biblatex \usepackage{tikz} \usepackage{pgfplots} +\usepackage{svg} +\usepackage{multirow} \usetikzlibrary{positioning} \usepackage{float} \usepackage{amsmath} @@ -106,15 +108,22 @@ resulting in error detection for 2 bits used, and error correction for 3+ bits u \end{minipage} \end{figure} - +Formally, a code can be understood as a function $C : \mathcal{X} \rightarrow \Sigma^*$ mapping data $\mathcal{X}$ to a string +(code word) over an alphabet set $\Sigma$. +With a code word $C(x)$, we can relate the length of data word to code word, resulting in the measure of information rate: +\[R = \frac{\log_q(|C|)}{n}\] +Given the size $q= |\Sigma|$ of an alphabet, size of a code $|C|$ and length of the code words $n$, +it relates the information to bits transmitted. +Because any error correction requires redundancy, i.e. adding bits to pure information, the \textit{information rate} has to decrease +with increasing error correction. +As a result, $R \leq 1$ ($<$ if error correction is used) and the redundancy is $1-R$. \subsection{Mathematical Bounds} In general, the amount of errors a code can detect or correct is determined by the Hamming distance $h$ defined as the number of positions in which neighboring strings (code words) differ. "Karolin" and "Kerstin" differ in 3 letters and thus have a Hamming distance of $h=3$, just as the binary example. -In general, a code is said to have a Hamming distance of $h$ if it is the minimal pairwise distance of all codewords. -Given a Hamming distance of $h$, $(h-1)$ errors can be detected, to correct $r$ errors a minimum distance of $h\geq 2r+1$ is required. +Given a Hamming distance of $h$, $(h-1)$ errors can be detected and to correct $r$ errors a minimum distance of $h\geq 2r+1$ is required. This is analogous to the repetition code already shown in \autoref{tab:detection-correction}, as the length of our repetition code is directly equivalent to the hamming distance. @@ -131,6 +140,8 @@ $\sum_{j=0}^{d-1} \binom{n}{j} (q-1)^j$, the size of a ball created around a cod The ball is the geometric interpretation of the space created around each used code word by requiring a minimal hamming distance, i.e. requiring that no code word lie closer within the vector space. +% TODO kraft mcmillan + \begin{figure}[h] \begin{minipage}{.3\textwidth} \begin{tabular}{c|c} @@ -147,8 +158,11 @@ i.e. requiring that no code word lie closer within the vector space. \section{Block Codes} -% NOTE fact-check Block codes are \textit{memoryless}, meaning that each block is encoded independently using a static dictionary. +Importantly, they can be described by a tuple $(n,k,d)$ taking in $k$ bits of information outputting $n$ bits of code word +while maintaining minimum distance $d$, +implying that the code words produced are of fixed length. + \subsection{Cyclic Redundancy Check} A \acrfull{crc} is a method of detecting (correcting) errors by interpreting the information to be sent as a polynomial. @@ -189,9 +203,31 @@ an even number of 1s. \end{minipage} \end{figure} +\subsection{Interleaving} +Another mitigation for burst errors is interleaving. +When strong burst errors occur for short intervals in a channel that is otherwise on average low distortion, +the idea is to spread out code words over time, i.e. interleave them in a pattern known to both transmitter and receiver. + +\[ +\begin{array}{cccc} + (x_{11} & x_{12} & x_{13} & x_{14}) \\ + (x_{21} & x_{22} & x_{23} & x_{24}) \\ + (x_{31} & x_{32} & x_{33} & x_{34}) \\ + (x_{41} & x_{42} & x_{43} & x_{44}) \\ +\end{array} +\rightarrow +x_{11} x_{21}x_{31}x_{41}x_{12}x_{22}x_{32}x_{42}x_{13}x_{23}x_{33}x_{43}x_{14}x_{24}x_{34}x_{44} +\] +In the above example, we interleave 4 code words $x_1$ through $x_4$ of length 4 by writing them row-wise in a table. +They are then read and transmitted columnwise, cycling bit by bit through each code word, leaving each of them less vulnerable to a short +burst error. +The receiver has to keep a buffer of 4 code words to de-interleave the transmission and then apply correction as usual. + \subsection{Reed-Solomon} +% TODO cyclic code Similar to \acrshort{crc}, a Reed-Solomon code also interprets the message as a polynomial. -This polynomial of degree $k-1$ is uniquely identified by $k$ evaluation points. +This polynomial of degree $k-1$ is uniquely identified by any $k$ evaluation points, +employing the same property as Shamir's secret sharing. By transmitting $n>k$ points, a Reed-Solomon code can detect $t=n-k$ errors or locate and correct up to $\lfloor t/2 \rfloor$ errors. Formally, the message will be $(a_1,a_2,...,a_k)$ coefficients of a polynomial @@ -200,7 +236,87 @@ f(x) = a_1 + a_2 x + a_3 x^2+...+a_k x^{k-1} = \sum_{i=1}^{k} a_i x^{i-1} \] \cite{enwiki:reed-solomon} +\clearpage \section{Convolutional Codes} +A convolutional code generates its parity symbols by using a sliding input window of a boolean polynomial function. +A visualisation is provided in \autoref{fig:convolutional}, where bit inputs are processed by a sliding window of size 3, +processed by 3 different functions resulting in a tuple $(C_1,C_2,C_3)$ being sent over the channel. +The rate of such a convolutional code is simply determined by the number of input bits to output bits, in this case $\frac{1}{3}$ +as our encoder will output 3 bits for every new input bit being shifted into the window. +\begin{figure}[H] + +\begin{minipage}{0.4\textwidth} + \centering + \includesvg[height=5cm]{figures/convolutional.drawio.svg} +\end{minipage} +\hfill +\begin{minipage}{0.4\textwidth} + \begin{tabular}{cc|ccc|c} + $S_0S_1$ & In & $C_1$ & $C_2$ & $C_3$ & $S_0'S_1'$ \\ + \hline + \multirow{2}{*}{00} & 0 & 0 & 0 & 0 & 00 \\ + & 1 & 1 & 0 & 1 & 10 \\ + \hline + \multirow{2}{*}{10} & 0 & 1 & 1 & 0 & 01 \\ + & 1 & 0 & 1 & 0 & 11 \\ + \hline + \multirow{2}{*}{01} & 0 & 1 & 1 & 0 & 00 \\ + & 1 & 0 & 1 & 0 & 10 \\ + \hline + \multirow{2}{*}{11} & 0 & 0 & 0 & 1 & 01\\ + & 1 & 1 & 0 & 0 & 11\\ + \end{tabular} +\end{minipage} + \caption{Visualisation and transitions of a convolutional encoder} + \label{fig:convolutional} +\end{figure} +Knowing the underlying function allows us to first create a table that later translates to a \textit{trellis}, +a data structure later used in decoding using the Viterbi algorithm. +Specifically, we can understand this window of 3 bits to be 2 bits of state $S_0S_1$ +and 1 bit of input, that is the leftmost bit in the window in \autoref{fig:convolutional}. + + + +\subsection{Viterbi algorithm} +The viterbi algorithm operates on a \textit{trellis}, that is a variant of a state machine that shows the transitions +from $S_0S_1$ to $S_0'S_1'$ like in \autoref{fig:trellis}. + +\begin{figure}[H] +\begin{tikzpicture} + % Draw the nodes + \node (00) at (0,3) [draw,circle] {00}; + \node (10) at (0,2) [draw,circle] {10}; + \node (01) at (0,1) [draw,circle] {01}; + \node (11) at (0,0) [draw,circle] {11}; + + \node (00') at (3,3) [draw,circle] {00}; + \node (10') at (3,2) [draw,circle] {10}; + \node (01') at (3,1) [draw,circle] {01}; + \node (11') at (3,0) [draw,circle] {11}; + + % Draw arrows between nodes + \draw[->] (00) -- (00') node[midway, above] {0} ; + \draw[->] (00) -- (10') node[pos=0.2, below] {1} ; + \draw[->] (10) -- (01') node[pos=0.2, above] {0} ; + \draw[->] (10) -- (11') node[midway, above] {1} ; + \draw[->] (01) -- (00') node[midway, above] {0} ; + \draw[->] (01) -- (10') node[pos=0.2, below] {1} ; + \draw[->] (11) -- (01') node[pos=0.2, above] {0} ; + \draw[->] (11) -- (11') node[midway, below] {1} ; +\end{tikzpicture} +\caption{Trellis for the viterbi algorithm} +\label{fig:trellis} +\end{figure} +For a given received sequence, we can then duplicate this trellis for each block of (in this example 3) bits +to decide which bits were most likely sent by minimizing the hamming distance through the graph. +As a result, we will decode a minimum error input message by retracing the path through the trellis. + +\section{Conclusion} +Numerous different forward error correction methods exist that optimise for different scenarios and vary in complexity. +In practice, they find applications in data storage and transmission, often in combination of each other, +with the most interesting implementations being used for extreme scenarios such as satellite communication. + +\clearpage \printbibliography \end{document} diff --git a/figures/convolutional.drawio.svg b/figures/convolutional.drawio.svg new file mode 100644 index 0000000..bb5249d --- /dev/null +++ b/figures/convolutional.drawio.svg @@ -0,0 +1,4 @@ + + + +
+
+
+
C1
C2
C3
 \ No newline at end of file