Files
handouts/src/Advanced/Error-Correcting Codes/parts/02 hamming.tex
Mark c29f0d25c7
All checks were successful
CI / Typos (pull_request) Successful in 7s
CI / Typst formatting (pull_request) Successful in 5s
CI / Build (pull_request) Successful in 4m32s
ECC edits
2025-03-02 19:41:20 -08:00

620 lines
17 KiB
TeX

\section{Hamming Codes}
Say we have an $n$-bit message, for example \texttt{1011 0101 1101 1001}. \par
We will number its bits in binary, from left to right:
\begin{center}
\begin{tikzpicture}
\node[anchor=west] at (-1.75, 0) {Bit};
\node[anchor=west] at (-1.75, -0.5) {Index};
\node at (0, 0) {\texttt{1}};
\node at (1, 0) {\texttt{0}};
\node at (2, 0) {\texttt{1}};
\node at (3, 0) {\texttt{1}};
\node at (4, 0) {\texttt{0}};
\node at (5, 0) {\texttt{1}};
\node at (6, 0) {\texttt{0}};
\node at (7, 0) {\texttt{1}};
\draw (-1.75, 0.25) -- (6.9, 0.25);
\draw (-1.75, -0.25) -- (6.9, -0.25);
\draw (-1.75, -0.75) -- (6.9, -0.75);
\foreach \x in {-1.75,-0.5,0.5,...,6.5} {
\draw (\x, 0.25) -- (\x, -0.75);
}
\node[color=gray] at (0, -0.5) {\texttt{0000}};
\node[color=gray] at (1, -0.5) {\texttt{0001}};
\node[color=gray] at (2, -0.5) {\texttt{0010}};
\node[color=gray] at (3, -0.5) {\texttt{0011}};
\node[color=gray] at (4, -0.5) {\texttt{0100}};
\node[color=gray] at (5, -0.5) {\texttt{0101}};
\node[color=gray] at (6, -0.5) {\texttt{0110}};
\node[color=gray] at (7, -0.5) {\texttt{0111}};
\draw[fill = white, draw = none]
(6.9, 0.25)
-- (7.1, 0)
-- (6.9, -0.25)
-- (7.1, -0.5)
-- (6.9, -0.75)
-- (7.5, -0.75)
-- (7.5, 0.25)
;
\draw (6.9, 0.25)
-- (7.1, 0)
-- (6.9, -0.25)
-- (7.1, -0.5)
-- (6.9, -0.75)
;
\node[anchor=west,color=gray] at (7.2, -0.25) { and so on...};
\end{tikzpicture}
\end{center}
\problem{}
If we number the bits of a 16-bit message as described above, \par
how many message bits have an index with a one as the last digit? \par
(i.e, an index that looks like \texttt{***1})
\vspace{2cm}
\problem{}
Now consider a 32-bit message. \par
How many message bits have an index with a one as the $n^\text{th}$ digit? \par
\vspace{2cm}
Now, let's come up with a way to detect errors in our 16-bit message.
To do this, we'll replace a few data bits with parity bits.
This will reduce the amount of information we can send,
but will allow the receiver to detect errors in the received message.
\vspace{1mm}
Let's arrange our message in a grid. We'll make the first bit (currently empty, marked \texttt{X}) a parity bit. Its value will depend on the content of the message: if our message has an even number of ones, it will be zero; if our message has an odd number of ones, it will be one. \par
This first bit ensures that there is an even number of ones in the whole message.
\begin{center}
\hfill
\begin{tikzpicture}[scale = 1.25]
\node at (0.75, 0.5) {Bit Numbering};
\node at (0.0, 0) {\texttt{0}};
\node at (0.5, 0) {\texttt{1}};
\node at (1.0, 0) {\texttt{2}};
\node at (1.5, 0) {\texttt{3}};
\node at (0.0, -0.5) {\texttt{4}};
\node at (0.5, -0.5) {\texttt{5}};
\node at (1.0, -0.5) {\texttt{6}};
\node at (1.5, -0.5) {\texttt{7}};
\node at (0.0, -1) {\texttt{8}};
\node at (0.5, -1) {\texttt{9}};
\node at (1.0, -1) {\texttt{10}};
\node at (1.5, -1) {\texttt{11}};
\node at (0.0, -1.5) {\texttt{12}};
\node at (0.5, -1.5) {\texttt{13}};
\node at (1.0, -1.5) {\texttt{14}};
\node at (1.5, -1.5) {\texttt{15}};
\draw (-0.25, 0.25) -- (1.75, 0.25);
\draw (-0.25, -0.25) -- (1.75, -0.25);
\draw (-0.25, -0.75) -- (1.75, -0.75);
\draw (-0.25, -1.25) -- (1.75, -1.25);
\draw (-0.25, -1.75) -- (1.75, -1.75);
\draw (-0.25, 0.25) -- (-0.25, -1.75);
\draw (0.25, 0.25) -- (0.25, -1.75);
\draw (0.75, 0.25) -- (0.75, -1.75);
\draw (1.25, 0.25) -- (1.25, -1.75);
\draw (1.75, 0.25) -- (1.75, -1.75);
\end{tikzpicture}
\hfill
\begin{tikzpicture}[scale = 1.25]
\node at (0.75, 0.5) {Sample Message};
\node at (0.0, 0) {\texttt{X}};
\node at (0.5, 0) {\texttt{0}};
\node at (1.0, 0) {\texttt{1}};
\node at (1.5, 0) {\texttt{1}};
\node at (0.0, -0.5) {\texttt{0}};
\node at (0.5, -0.5) {\texttt{1}};
\node at (1.0, -0.5) {\texttt{0}};
\node at (1.5, -0.5) {\texttt{1}};
\node at (0.0, -1) {\texttt{1}};
\node at (0.5, -1) {\texttt{1}};
\node at (1.0, -1) {\texttt{0}};
\node at (1.5, -1) {\texttt{1}};
\node at (0.0, -1.5) {\texttt{1}};
\node at (0.5, -1.5) {\texttt{0}};
\node at (1.0, -1.5) {\texttt{0}};
\node at (1.5, -1.5) {\texttt{1}};
\draw (-0.25, 0.25) -- (1.75, 0.25);
\draw (-0.25, -0.25) -- (1.75, -0.25);
\draw (-0.25, -0.75) -- (1.75, -0.75);
\draw (-0.25, -1.25) -- (1.75, -1.25);
\draw (-0.25, -1.75) -- (1.75, -1.75);
\draw (-0.25, 0.25) -- (-0.25, -1.75);
\draw (0.25, 0.25) -- (0.25, -1.75);
\draw (0.75, 0.25) -- (0.75, -1.75);
\draw (1.25, 0.25) -- (1.25, -1.75);
\draw (1.75, 0.25) -- (1.75, -1.75);
\draw (-0.2,-0.2) -- (0.2, -0.2) -- (0.2, 0.2) -- (-0.2, 0.2) -- (-0.2,-0.2);
\end{tikzpicture}
\hfill~
\end{center}
\problem{}
What is the value of the parity bit in the message above?
\vfill
\problem{}
Can this coding scheme detect a transposition error? \par
Can this coding scheme detect two single-bit errors? \par
Can this coding scheme correct a single-bit error?
\vfill
\pagebreak
We'll now add four more parity bits, in positions \texttt{0001}, \texttt{0010}, \texttt{0100}, and \texttt{1000}: \par
This error-detection scheme is called the \textit{Hamming code}.
\begin{center}
\begin{tikzpicture}[scale = 1.25]
\node at (0.0, 0) {\texttt{X}};
\node at (0.5, 0) {\texttt{X}};
\node at (1.0, 0) {\texttt{X}};
\node at (1.5, 0) {\texttt{1}};
\node at (0.0, -0.5) {\texttt{X}};
\node at (0.5, -0.5) {\texttt{1}};
\node at (1.0, -0.5) {\texttt{0}};
\node at (1.5, -0.5) {\texttt{1}};
\node at (0.0, -1) {\texttt{X}};
\node at (0.5, -1) {\texttt{1}};
\node at (1.0, -1) {\texttt{0}};
\node at (1.5, -1) {\texttt{1}};
\node at (0.0, -1.5) {\texttt{1}};
\node at (0.5, -1.5) {\texttt{0}};
\node at (1.0, -1.5) {\texttt{0}};
\node at (1.5, -1.5) {\texttt{1}};
\draw (-0.25, 0.25) -- (1.75, 0.25);
\draw (-0.25, -0.25) -- (1.75, -0.25);
\draw (-0.25, -0.75) -- (1.75, -0.75);
\draw (-0.25, -1.25) -- (1.75, -1.25);
\draw (-0.25, -1.75) -- (1.75, -1.75);
\draw (-0.25, 0.25) -- (-0.25, -1.75);
\draw (0.25, 0.25) -- (0.25, -1.75);
\draw (0.75, 0.25) -- (0.75, -1.75);
\draw (1.25, 0.25) -- (1.25, -1.75);
\draw (1.75, 0.25) -- (1.75, -1.75);
\draw (0 - 0.2, 0 - 0.2)
-- (0 + 0.2, 0 - 0.2)
-- (0 + 0.2, 0 + 0.2)
-- (0 - 0.2, 0 + 0.2)
-- (0 - 0.2, 0 - 0.2);
\draw (0.5 - 0.2, 0 - 0.2)
-- (0.5 + 0.2, 0 - 0.2)
-- (0.5 + 0.2, 0 + 0.2)
-- (0.5 - 0.2, 0 + 0.2)
-- (0.5 - 0.2, 0 - 0.2);
\draw (1 - 0.2, 0 - 0.2)
-- (1 + 0.2, 0 - 0.2)
-- (1 + 0.2, 0 + 0.2)
-- (1 - 0.2, 0 + 0.2)
-- (1 - 0.2, 0 - 0.2);
\draw (0 - 0.2, -0.5 - 0.2)
-- (0 + 0.2, -0.5 - 0.2)
-- (0 + 0.2, -0.5 + 0.2)
-- (0 - 0.2, -0.5 + 0.2)
-- (0 - 0.2, -0.5 - 0.2);
\draw (0 - 0.2, -1 - 0.2)
-- (0 + 0.2, -1 - 0.2)
-- (0 + 0.2, -1 + 0.2)
-- (0 - 0.2, -1 + 0.2)
-- (0 - 0.2, -1 - 0.2);
\end{tikzpicture}
\end{center}
Bit \texttt{0001} will count the parity of all bits with a one in the first digit of their index. \par
Bit \texttt{0010} will count the parity of all bits with a one in the second digit of their index. \par
Bits \texttt{0100} and \texttt{1000} work in the same way. \par
\hint{In \texttt{0001}, \texttt{1} is the first digit. In \texttt{0010}, \texttt{1} is the second digit. \\
When counting bits in binary numbers, go from right to left.}
\problem{}
Which message bits does each parity bit \say{cover}? \par
In other words, which message bits affect the value of each parity bit? \par
\vspace{1mm}
Four diagrams are shown below. In each grid, fill in the bits that affect the shaded parity bit.
\begin{center}
\hfill
\begin{tikzpicture}[scale = 1.25]
\draw (-0.25, 0.25) -- (1.75, 0.25);
\draw (-0.25, -0.25) -- (1.75, -0.25);
\draw (-0.25, -0.75) -- (1.75, -0.75);
\draw (-0.25, -1.25) -- (1.75, -1.25);
\draw (-0.25, -1.75) -- (1.75, -1.75);
\draw (-0.25, 0.25) -- (-0.25, -1.75);
\draw (0.25, 0.25) -- (0.25, -1.75);
\draw (0.75, 0.25) -- (0.75, -1.75);
\draw (1.25, 0.25) -- (1.25, -1.75);
\draw (1.75, 0.25) -- (1.75, -1.75);
\draw[pattern=north east lines] (0.5 - 0.2, 0 - 0.2)
-- (0.5 + 0.2, 0 - 0.2)
-- (0.5 + 0.2, 0 + 0.2)
-- (0.5 - 0.2, 0 + 0.2)
-- (0.5 - 0.2, 0 - 0.2);
\end{tikzpicture}
\hfill
\begin{tikzpicture}[scale = 1.25]
\draw (-0.25, 0.25) -- (1.75, 0.25);
\draw (-0.25, -0.25) -- (1.75, -0.25);
\draw (-0.25, -0.75) -- (1.75, -0.75);
\draw (-0.25, -1.25) -- (1.75, -1.25);
\draw (-0.25, -1.75) -- (1.75, -1.75);
\draw (-0.25, 0.25) -- (-0.25, -1.75);
\draw (0.25, 0.25) -- (0.25, -1.75);
\draw (0.75, 0.25) -- (0.75, -1.75);
\draw (1.25, 0.25) -- (1.25, -1.75);
\draw (1.75, 0.25) -- (1.75, -1.75);
\draw[pattern=north east lines] (1 - 0.2, 0 - 0.2)
-- (1 + 0.2, 0 - 0.2)
-- (1 + 0.2, 0 + 0.2)
-- (1 - 0.2, 0 + 0.2)
-- (1 - 0.2, 0 - 0.2);
\end{tikzpicture}
\hfill
\begin{tikzpicture}[scale = 1.25]
\draw (-0.25, 0.25) -- (1.75, 0.25);
\draw (-0.25, -0.25) -- (1.75, -0.25);
\draw (-0.25, -0.75) -- (1.75, -0.75);
\draw (-0.25, -1.25) -- (1.75, -1.25);
\draw (-0.25, -1.75) -- (1.75, -1.75);
\draw (-0.25, 0.25) -- (-0.25, -1.75);
\draw (0.25, 0.25) -- (0.25, -1.75);
\draw (0.75, 0.25) -- (0.75, -1.75);
\draw (1.25, 0.25) -- (1.25, -1.75);
\draw (1.75, 0.25) -- (1.75, -1.75);
\draw[pattern=north east lines] (0 - 0.2, -0.5 - 0.2)
-- (0 + 0.2, -0.5 - 0.2)
-- (0 + 0.2, -0.5 + 0.2)
-- (0 - 0.2, -0.5 + 0.2)
-- (0 - 0.2, -0.5 - 0.2);
\end{tikzpicture}
\hfill
\begin{tikzpicture}[scale = 1.25]
\draw (-0.25, 0.25) -- (1.75, 0.25);
\draw (-0.25, -0.25) -- (1.75, -0.25);
\draw (-0.25, -0.75) -- (1.75, -0.75);
\draw (-0.25, -1.25) -- (1.75, -1.25);
\draw (-0.25, -1.75) -- (1.75, -1.75);
\draw (-0.25, 0.25) -- (-0.25, -1.75);
\draw (0.25, 0.25) -- (0.25, -1.75);
\draw (0.75, 0.25) -- (0.75, -1.75);
\draw (1.25, 0.25) -- (1.25, -1.75);
\draw (1.75, 0.25) -- (1.75, -1.75);
\draw[pattern=north east lines] (0 - 0.2, -1 - 0.2)
-- (0 + 0.2, -1 - 0.2)
-- (0 + 0.2, -1 + 0.2)
-- (0 - 0.2, -1 + 0.2)
-- (0 - 0.2, -1 - 0.2);
\end{tikzpicture}
\hfill
\end{center}
\problem{}
Compute all parity bits in the message above.
\vfill
\pagebreak
\problem{}
Analyze this coding scheme.
\begin{itemize}
\item Can we detect one single-bit errors?
\item Can we detect two single-bit errors?
\item What errors can we correct?
\end{itemize}
\vfill
\problem{}
Each of the following messages has either one, two, or no errors. \par
Find the errors and correct them if possible. \par
\hint{Bit \texttt{0000} should tell you how many errors you have.}
\begin{center}
\hfill
\begin{tikzpicture}[scale = 1.25]
\node at (0.0, 0) {\texttt{0}};
\node at (0.5, 0) {\texttt{1}};
\node at (1.0, 0) {\texttt{1}};
\node at (1.5, 0) {\texttt{1}};
\node at (0.0, -0.5) {\texttt{0}};
\node at (0.5, -0.5) {\texttt{1}};
\node at (1.0, -0.5) {\texttt{1}};
\node at (1.5, -0.5) {\texttt{1}};
\node at (0.0, -1) {\texttt{0}};
\node at (0.5, -1) {\texttt{0}};
\node at (1.0, -1) {\texttt{1}};
\node at (1.5, -1) {\texttt{1}};
\node at (0.0, -1.5) {\texttt{1}};
\node at (0.5, -1.5) {\texttt{1}};
\node at (1.0, -1.5) {\texttt{1}};
\node at (1.5, -1.5) {\texttt{0}};
\draw (-0.25, 0.25) -- (1.75, 0.25);
\draw (-0.25, -0.25) -- (1.75, -0.25);
\draw (-0.25, -0.75) -- (1.75, -0.75);
\draw (-0.25, -1.25) -- (1.75, -1.25);
\draw (-0.25, -1.75) -- (1.75, -1.75);
\draw (-0.25, 0.25) -- (-0.25, -1.75);
\draw (0.25, 0.25) -- (0.25, -1.75);
\draw (0.75, 0.25) -- (0.75, -1.75);
\draw (1.25, 0.25) -- (1.25, -1.75);
\draw (1.75, 0.25) -- (1.75, -1.75);
\draw (0 - 0.2, 0 - 0.2)
-- (0 + 0.2, 0 - 0.2)
-- (0 + 0.2, 0 + 0.2)
-- (0 - 0.2, 0 + 0.2)
-- (0 - 0.2, 0 - 0.2);
\draw (0.5 - 0.2, 0 - 0.2)
-- (0.5 + 0.2, 0 - 0.2)
-- (0.5 + 0.2, 0 + 0.2)
-- (0.5 - 0.2, 0 + 0.2)
-- (0.5 - 0.2, 0 - 0.2);
\draw (1 - 0.2, 0 - 0.2)
-- (1 + 0.2, 0 - 0.2)
-- (1 + 0.2, 0 + 0.2)
-- (1 - 0.2, 0 + 0.2)
-- (1 - 0.2, 0 - 0.2);
\draw (0 - 0.2, -0.5 - 0.2)
-- (0 + 0.2, -0.5 - 0.2)
-- (0 + 0.2, -0.5 + 0.2)
-- (0 - 0.2, -0.5 + 0.2)
-- (0 - 0.2, -0.5 - 0.2);
\draw (0 - 0.2, -1 - 0.2)
-- (0 + 0.2, -1 - 0.2)
-- (0 + 0.2, -1 + 0.2)
-- (0 - 0.2, -1 + 0.2)
-- (0 - 0.2, -1 - 0.2);
\end{tikzpicture}
\hfill
\begin{tikzpicture}[scale = 1.25]
\node at (0.0, 0) {\texttt{1}};
\node at (0.5, 0) {\texttt{1}};
\node at (1.0, 0) {\texttt{0}};
\node at (1.5, 0) {\texttt{1}};
\node at (0.0, -0.5) {\texttt{1}};
\node at (0.5, -0.5) {\texttt{0}};
\node at (1.0, -0.5) {\texttt{1}};
\node at (1.5, -0.5) {\texttt{0}};
\node at (0.0, -1) {\texttt{0}};
\node at (0.5, -1) {\texttt{1}};
\node at (1.0, -1) {\texttt{1}};
\node at (1.5, -1) {\texttt{0}};
\node at (0.0, -1.5) {\texttt{1}};
\node at (0.5, -1.5) {\texttt{1}};
\node at (1.0, -1.5) {\texttt{0}};
\node at (1.5, -1.5) {\texttt{1}};
\draw (-0.25, 0.25) -- (1.75, 0.25);
\draw (-0.25, -0.25) -- (1.75, -0.25);
\draw (-0.25, -0.75) -- (1.75, -0.75);
\draw (-0.25, -1.25) -- (1.75, -1.25);
\draw (-0.25, -1.75) -- (1.75, -1.75);
\draw (-0.25, 0.25) -- (-0.25, -1.75);
\draw (0.25, 0.25) -- (0.25, -1.75);
\draw (0.75, 0.25) -- (0.75, -1.75);
\draw (1.25, 0.25) -- (1.25, -1.75);
\draw (1.75, 0.25) -- (1.75, -1.75);
\draw (0 - 0.2, 0 - 0.2)
-- (0 + 0.2, 0 - 0.2)
-- (0 + 0.2, 0 + 0.2)
-- (0 - 0.2, 0 + 0.2)
-- (0 - 0.2, 0 - 0.2);
\draw (0.5 - 0.2, 0 - 0.2)
-- (0.5 + 0.2, 0 - 0.2)
-- (0.5 + 0.2, 0 + 0.2)
-- (0.5 - 0.2, 0 + 0.2)
-- (0.5 - 0.2, 0 - 0.2);
\draw (1 - 0.2, 0 - 0.2)
-- (1 + 0.2, 0 - 0.2)
-- (1 + 0.2, 0 + 0.2)
-- (1 - 0.2, 0 + 0.2)
-- (1 - 0.2, 0 - 0.2);
\draw (0 - 0.2, -0.5 - 0.2)
-- (0 + 0.2, -0.5 - 0.2)
-- (0 + 0.2, -0.5 + 0.2)
-- (0 - 0.2, -0.5 + 0.2)
-- (0 - 0.2, -0.5 - 0.2);
\draw (0 - 0.2, -1 - 0.2)
-- (0 + 0.2, -1 - 0.2)
-- (0 + 0.2, -1 + 0.2)
-- (0 - 0.2, -1 + 0.2)
-- (0 - 0.2, -1 - 0.2);
\end{tikzpicture}
\hfill
\begin{tikzpicture}[scale = 1.25]
\node at (0.0, 0) {\texttt{0}};
\node at (0.5, 0) {\texttt{1}};
\node at (1.0, 0) {\texttt{1}};
\node at (1.5, 0) {\texttt{1}};
\node at (0.0, -0.5) {\texttt{1}};
\node at (0.5, -0.5) {\texttt{0}};
\node at (1.0, -0.5) {\texttt{1}};
\node at (1.5, -0.5) {\texttt{1}};
\node at (0.0, -1) {\texttt{1}};
\node at (0.5, -1) {\texttt{0}};
\node at (1.0, -1) {\texttt{1}};
\node at (1.5, -1) {\texttt{1}};
\node at (0.0, -1.5) {\texttt{1}};
\node at (0.5, -1.5) {\texttt{0}};
\node at (1.0, -1.5) {\texttt{0}};
\node at (1.5, -1.5) {\texttt{0}};
\draw (-0.25, 0.25) -- (1.75, 0.25);
\draw (-0.25, -0.25) -- (1.75, -0.25);
\draw (-0.25, -0.75) -- (1.75, -0.75);
\draw (-0.25, -1.25) -- (1.75, -1.25);
\draw (-0.25, -1.75) -- (1.75, -1.75);
\draw (-0.25, 0.25) -- (-0.25, -1.75);
\draw (0.25, 0.25) -- (0.25, -1.75);
\draw (0.75, 0.25) -- (0.75, -1.75);
\draw (1.25, 0.25) -- (1.25, -1.75);
\draw (1.75, 0.25) -- (1.75, -1.75);
\draw (0 - 0.2, 0 - 0.2)
-- (0 + 0.2, 0 - 0.2)
-- (0 + 0.2, 0 + 0.2)
-- (0 - 0.2, 0 + 0.2)
-- (0 - 0.2, 0 - 0.2);
\draw (0.5 - 0.2, 0 - 0.2)
-- (0.5 + 0.2, 0 - 0.2)
-- (0.5 + 0.2, 0 + 0.2)
-- (0.5 - 0.2, 0 + 0.2)
-- (0.5 - 0.2, 0 - 0.2);
\draw (1 - 0.2, 0 - 0.2)
-- (1 + 0.2, 0 - 0.2)
-- (1 + 0.2, 0 + 0.2)
-- (1 - 0.2, 0 + 0.2)
-- (1 - 0.2, 0 - 0.2);
\draw (0 - 0.2, -0.5 - 0.2)
-- (0 + 0.2, -0.5 - 0.2)
-- (0 + 0.2, -0.5 + 0.2)
-- (0 - 0.2, -0.5 + 0.2)
-- (0 - 0.2, -0.5 - 0.2);
\draw (0 - 0.2, -1 - 0.2)
-- (0 + 0.2, -1 - 0.2)
-- (0 + 0.2, -1 + 0.2)
-- (0 - 0.2, -1 + 0.2)
-- (0 - 0.2, -1 - 0.2);
\end{tikzpicture}
\hfill
\end{center}
\begin{solution}
\textbf{1:} Single error at position \texttt{1010} \par
\textbf{2:} Double error \par
\textbf{3:} No error \par
\end{solution}
\vfill
\pagebreak
\problem{}
How many parity bits does each message bit affect? \par
Does this correlate with that message bit's index?
\vfill
\problem{}
Say we have a message with exactly one single-bit error. \par
If we know which parity bits are inconsistent, how can we find where the error is?
\vfill
\problem{}<generalize-hamming>
Generalize this system for messages of 4, 64, or 256 bits. \par
\begin{itemize}
\item How does the resilience of this scheme change if we use a larger message size?
\item How does the efficiency of this scheme change if we send larger messages?
\end{itemize}
\vfill
\pagebreak
\definition{}
A \textit{deletion} error occurs when one bit of the message is deleted. \par
Likewise, an \textit{insertion} error consists of a random inserted bit. \par
\definition{}
A \textit{message stream} is an infinite string of binary digits.
\problem{}
Show that Hamming codes do not reliably detect bit deletions: \par
\hint{
Create a 17-bit message whose first 16 bits are a valid Hamming-coded message, \par
and which is still valid when a bit (chosen by you; not the $17^\text{th}$) is deleted.
}
\vfill
\problem{}
Convince yourself that Hamming codes cannot correct insertions. \par
Then, create a 16-bit message that:
\begin{itemize}
\item is itself a valid Hamming code, and
\item incorrectly "corrects" a single bit error when it encounters an insertion error. \par
You may choose where the insertion occurs.
\end{itemize}
\vfill
As we have seen, Hamming codes effectively handle substitutions, but cannot reliably
detect (or correct) insertions and deletions. Correcting those errors is a bit more difficult:
if the number of bits we receive is variable, how can we split a stream into a series of messages? \par
\note{This is a rhetorical question, which we'll discuss another day.}
\pagebreak