convolutional codes

correction.tex

@@ -9,6 +9,8 @@
 %\usepackage{csquotes} % Recommended for biblatex
 \usepackage{tikz}
 \usepackage{pgfplots}
+\usepackage{svg}
+\usepackage{multirow}
 \usetikzlibrary{positioning}
 \usepackage{float}
 \usepackage{amsmath}
@@ -106,15 +108,22 @@ resulting in error detection for 2 bits used, and error correction for 3+ bits u
 \end{minipage}
 \end{figure}
 
-
+Formally, a code can be understood as a function $C : \mathcal{X} \rightarrow \Sigma^*$ mapping data $\mathcal{X}$ to a string
+(code word) over an alphabet set $\Sigma$.
+With a code word $C(x)$, we can relate the length of data word to code word, resulting in the measure of information rate:
+\[R = \frac{\log_q(|C|)}{n}\]
+Given the size $q= |\Sigma|$ of an alphabet, size of a code $|C|$ and length of the code words $n$,
+it relates the information to bits transmitted.
+Because any error correction requires redundancy, i.e. adding bits to pure information, the \textit{information rate} has to decrease
+with increasing error correction.
+As a result, $R \leq 1$ ($<$ if error correction is used) and the redundancy is $1-R$.
 
 \subsection{Mathematical Bounds}
 In general, the amount of errors a code can detect or correct
 is determined by the Hamming distance $h$ defined as the number of positions in which neighboring strings (code words) differ.
 
 "Karolin" and "Kerstin" differ in 3 letters and thus have a Hamming distance of $h=3$, just as the binary example.
-In general, a code is said to have a Hamming distance of $h$ if it is the minimal pairwise distance of all codewords.
-Given a Hamming distance of $h$, $(h-1)$ errors can be detected, to correct $r$ errors a minimum distance of $h\geq 2r+1$ is required.
+Given a Hamming distance of $h$, $(h-1)$ errors can be detected and to correct $r$ errors a minimum distance of $h\geq 2r+1$ is required.
 This is analogous to the repetition code already shown in \autoref{tab:detection-correction},
 as the length of our repetition code is directly equivalent to the hamming distance.
 
@@ -131,6 +140,8 @@ $\sum_{j=0}^{d-1} \binom{n}{j} (q-1)^j$, the size of a ball created around a cod
 The ball is the geometric interpretation of the space created around each used code word by requiring a minimal hamming distance,
 i.e. requiring that no code word lie closer within the vector space.
 
+% TODO kraft mcmillan
+
 \begin{figure}[h]
   \begin{minipage}{.3\textwidth}
     \begin{tabular}{c|c}
@@ -147,8 +158,11 @@ i.e. requiring that no code word lie closer within the vector space.
 
 
 \section{Block Codes}
-% NOTE fact-check
 Block codes are \textit{memoryless}, meaning that each block is encoded independently using a static dictionary.
+Importantly, they can be described by a tuple $(n,k,d)$ taking in $k$ bits of information outputting $n$ bits of code word
+while maintaining minimum distance $d$,
+implying that the code words produced are of fixed length.
+
 
 \subsection{Cyclic Redundancy Check}
 A \acrfull{crc} is a method of detecting (correcting) errors by interpreting the information to be sent as a polynomial.
@@ -189,9 +203,31 @@ an even number of 1s.
 \end{minipage}
 \end{figure}
 
+\subsection{Interleaving}
+Another mitigation for burst errors is interleaving.
+When strong burst errors occur for short intervals in a channel that is otherwise on average low distortion,
+the idea is to spread out code words over time, i.e. interleave them in a pattern known to both transmitter and receiver.
+  
+\[
+\begin{array}{cccc}
+  (x_{11} & x_{12} & x_{13} & x_{14}) \\
+  (x_{21} & x_{22} & x_{23} & x_{24}) \\
+  (x_{31} & x_{32} & x_{33} & x_{34}) \\
+  (x_{41} & x_{42} & x_{43} & x_{44}) \\
+\end{array}
+\rightarrow
+x_{11} x_{21}x_{31}x_{41}x_{12}x_{22}x_{32}x_{42}x_{13}x_{23}x_{33}x_{43}x_{14}x_{24}x_{34}x_{44}
+\]
+In the above example, we interleave 4 code words $x_1$ through $x_4$ of length 4 by writing them row-wise in a table.
+They are then read and transmitted columnwise, cycling bit by bit through each code word, leaving each of them less vulnerable to a short
+burst error.
+The receiver has to keep a buffer of 4 code words to de-interleave the transmission and then apply correction as usual.
+
 \subsection{Reed-Solomon}
+% TODO cyclic code
 Similar to \acrshort{crc}, a Reed-Solomon code also interprets the message as a polynomial.
-This polynomial of degree $k-1$ is uniquely identified by $k$ evaluation points.
+This polynomial of degree $k-1$ is uniquely identified by any $k$ evaluation points,
+employing the same property as Shamir's secret sharing.
 By transmitting $n>k$ points, a Reed-Solomon code can detect $t=n-k$ errors or locate and correct up to $\lfloor t/2 \rfloor$ errors.
 
 Formally, the message will be $(a_1,a_2,...,a_k)$ coefficients of a polynomial
@@ -200,7 +236,87 @@ f(x) = a_1 + a_2 x + a_3 x^2+...+a_k x^{k-1} = \sum_{i=1}^{k} a_i x^{i-1}
 \]
 \cite{enwiki:reed-solomon}
 
+\clearpage
 \section{Convolutional Codes}
+A convolutional code generates its parity symbols by using a sliding input window of a boolean polynomial function.
+A visualisation is provided in \autoref{fig:convolutional}, where bit inputs are processed by a sliding window of size 3,
+processed by 3 different functions resulting in a tuple $(C_1,C_2,C_3)$ being sent over the channel.
+The rate of such a convolutional code is simply determined by the number of input bits to output bits, in this case $\frac{1}{3}$
+as our encoder will output 3 bits for every new input bit being shifted into the window.
+\begin{figure}[H]
   
+\begin{minipage}{0.4\textwidth}
+    \centering
+    \includesvg[height=5cm]{figures/convolutional.drawio.svg}
+\end{minipage}
+\hfill
+\begin{minipage}{0.4\textwidth}
+  \begin{tabular}{cc|ccc|c}
+    $S_0S_1$ & In & $C_1$ & $C_2$ & $C_3$ & $S_0'S_1'$ \\
+    \hline
+    \multirow{2}{*}{00} & 0 & 0 & 0 & 0 & 00 \\
+                        & 1 & 1 & 0 & 1 & 10 \\
+    \hline
+    \multirow{2}{*}{10} & 0 & 1 & 1 & 0 & 01 \\
+                        & 1 & 0 & 1 & 0 & 11 \\
+    \hline
+    \multirow{2}{*}{01} & 0 & 1 & 1 & 0 & 00 \\
+                        & 1 & 0 & 1 & 0 & 10 \\
+    \hline
+    \multirow{2}{*}{11} & 0 & 0 & 0 & 1 & 01\\
+                        & 1 & 1 & 0 & 0 & 11\\
+  \end{tabular}
+\end{minipage}
+    \caption{Visualisation and transitions of a convolutional encoder}
+    \label{fig:convolutional}
+\end{figure}
+Knowing the underlying function allows us to first create a table that later translates to a \textit{trellis},
+a data structure later used in decoding using the Viterbi algorithm.
+
+Specifically, we can understand this window of 3 bits to be 2 bits of state $S_0S_1$
+and 1 bit of input, that is the leftmost bit in the window in \autoref{fig:convolutional}.
+
+
+
+\subsection{Viterbi algorithm}
+The viterbi algorithm operates on a \textit{trellis}, that is a variant of a state machine that shows the transitions
+from $S_0S_1$ to $S_0'S_1'$ like in \autoref{fig:trellis}.
+
+\begin{figure}[H]
+\begin{tikzpicture}
+    % Draw the nodes
+    \node (00) at (0,3) [draw,circle] {00};
+    \node (10) at (0,2) [draw,circle] {10};
+    \node (01) at (0,1) [draw,circle] {01};
+    \node (11) at (0,0) [draw,circle] {11};
+
+    \node (00') at (3,3) [draw,circle] {00};
+    \node (10') at (3,2) [draw,circle] {10};
+    \node (01') at (3,1) [draw,circle] {01};
+    \node (11') at (3,0) [draw,circle] {11};
+    
+    % Draw arrows between nodes
+    \draw[->] (00) -- (00') node[midway, above] {0} ;
+    \draw[->] (00) -- (10') node[pos=0.2, below] {1} ;
+    \draw[->] (10) -- (01') node[pos=0.2, above] {0} ;
+    \draw[->] (10) -- (11') node[midway, above] {1} ;
+    \draw[->] (01) -- (00') node[midway, above] {0} ;
+    \draw[->] (01) -- (10') node[pos=0.2, below] {1} ;
+    \draw[->] (11) -- (01') node[pos=0.2, above] {0} ;
+    \draw[->] (11) -- (11') node[midway, below] {1} ;
+\end{tikzpicture}
+\caption{Trellis for the viterbi algorithm}
+\label{fig:trellis}
+\end{figure}
+For a given received sequence, we can then duplicate this trellis for each block of (in this example 3) bits
+to decide which bits were most likely sent by minimizing the hamming distance through the graph.
+As a result, we will decode a minimum error input message by retracing the path through the trellis.
+
+\section{Conclusion}
+Numerous different forward error correction methods exist that optimise for different scenarios and vary in complexity.
+In practice, they find applications in data storage and transmission, often in combination of each other,
+with the most interesting implementations being used for extreme scenarios such as satellite communication.
+
+\clearpage
 \printbibliography
 \end{document}