See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/220613378

An application of vector space theory in data transmission

Article  in  ACM SIGCSE Bulletin · June 2006

DOI: 10.1145/1138403.1138428 · Source: DBLP

CITATIONS READS
0 6,168

1 author:

Liejune Shiau
University of Houston - Clear Lake
25 PUBLICATIONS   377 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Liejune Shiau on 24 March 2014.

The user has requested enhancement of the downloaded file.

 Reviewed Papers

An Application of Vector Space Theory

in Data Transmission

LieJune Shiau
University of Houston-Clear Lake
2700 Bay Area Boulevard
Houston, Texas 77058 USA
[email protected]

Abstract
This work illustrates an application of vector spaces to data transmission theory. We show how Hamming code error
detection and error correction are done through the tool of various theories in vector space. It is hoped that this article
will explain the importance of abstract mathematics, such as vector space and basis, in the application of data
transmission, which enlightens mathematics and computer science majors.

Keywords: vector space, data transmission, code words, code space

1. Introduction
The purpose of this article is to introduce the reader to an Sending Coded Message
interesting real world application of vector spaces. We
Binary Message Encoder
wish to exhibit one of the many strong links between
abstract and applied mathematics. The ubiquity of the Noise Channel
Internet makes data transmission a particularly relevant
“real world” problem. Computer data is transmitted in a Binary Message Decoder
binary mode and hence is represented in strings of 0’s and
Receiving Coded Message
1’s called bits. When transmitting from a home computer
via a modem, data is susceptible to “noise” corruption by Figure 1: Data Communication Model
an electromotive force, or to loss of signal due to
attenuation. Corruption of the binary signal simply means
some 0’s are changed into 1’s and vice versa. To In spite of the obvious limitations, the single parity
counteract such corruption Engineers utilize data encoders check bit scheme is often use in low speed, non-critical
and decoders. The diagram in Figure 1 shows a typical applications. When a higher degree of accuracy is needed,
digital data communication system. engineers turn to Hamming Code Theory to find and
The most basic method of error detection utilizes a correct errors. The Hamming Code was devised by
single-parity-check-bit scheme. An encoder simply Richard Hamming in late 1940’s and is currently the
attaches an extra binary bit, called a parity check bit, to the predominant error detection and correction method in use.
end of the message. The parity check bit can be computed What we propose to explain the process by which
the sum of all message bits (even parity check). When the Hamming Code error correction finds and corrects a single
decoder receives a string, it computes the binary sum of all transmission error. It will be seen that vector spaces
bits, including the parity check bit. If there are an odd provide the natural language in which to explain the
number of errors, the sum will be 1; if there are no errors, process.
or an even number of errors, the sum will be 0. The Hamming Code utilizes the idea of multiple parity
weakness of this method is that only an odd number of check bits. The sole purpose of the parity check bits is to
errors can be detected, and the location of the error remains detect the location of transmission errors in the message.
unknown. This means we cannot fix the errors, and instead Since we are encoding data in a binary fashion, any
the decoder must request a re-send of the data. corrupted bit is simply corrected by replacing the value
with its complement (i.e. 0’s are replaced with 1’s and vice
versa.) A major concern in the design of the encoding
scheme is that the parity check bits will provide the

inroads – The SIGCSE Bulletin 33 Volume 38, Number 2, 2006 June

 Reviewed Papers

decoder with enough information to accurately correct  c1 

transmission errors [3]. Parity check bits are simply c 
 2
appended to the data being transmitted. For example, the 1 1 0 1 1 0 0 c3  0
transmission of a k bit message will actually require that n 1 0 1 1 0 1 0 c  = 0
bits be transmitted, with n-k parity check bits added to the   4   
message, as shown in Figure 2. 0 1 1 1 0 0 1 c5  0
 
It can be seen that the string of message bits, with c6 
c 
parity check bits appended, form the elements of a vector  7
n
space. Let ∧ 2 be the set of all binary n-tuples of numbers
n
Define H to be the left matrix in the equation, i.e.
from ∧2 = {0,1}. ∧ 2 is a vector space over ∧2 by the usual
modulo-2 addition and scalar multiplication operations. In 1 1 0 1 1 0 0
n
∧2 , let e1 = (1,0,..., 0), e2 = (0, 1, 0,..., 0),..., en = (0,..., 0, H = 1 0 1 1 0 1 0
n 0 1 1 1 0 0 1
1). We know {e1, e2,..., en} as the standard basis for ∧ 2
n
and hence ∧ 2 is an n-dimensional vector space [2]. Our We call H the parity check matrix, because the rows of H
goal is to explain Hamming Code theory in terms of this are derived from the parity check equations. In this design,
vector space. We will explain in detail how Hamming any code word c in the code word set, denoted as C7,4
Code theory allows us to encode a message so that a single 7
which is a subset of ∧ 2 , satisfies Hc = 0. This means any
transmission error can be detected and corrected.
string c is in the code word set C7,4 if and only if c is in the
Null Space of H. Thus, C7,4 = N.S.(H). The code word set
encoder C7,4 is therefore called the code space, which is the vector
Message Message| Parity Checks
space of all correctly sent code words. It is impossible for
a single transmission error to change one code word into
another code word. We explain why with the use of a code
{ k } decoder { k, n-k } space metric function, called the Hamming Distance. In
our design, there are only 16 possible code words in the
Figure 2: Bit Assignment
code space C7,4. We list them all as follows:

(0, 0, 0, 0, 0, 0, 0) (1, 0, 0, 0, 1, 1, 0) (0, 1, 0, 0, 1, 0, 1)

Consider the task of transmitting a 4-bit message. We (0, 0, 1, 0, 0, 1, 1) (0, 0, 0, 1, 1, 1, 1)
will see that Hamming Code requires us to add 3 parity (1, 1, 0, 0, 0, 1, 1) (1, 0, 1, 0, 1, 0, 1) (1, 0, 0, 1, 0, 0, 1)
check bits in order to be able to correct a single (0, 1, 1, 0, 1, 1, 0) (0, 1, 0, 1, 0, 1, 0)
transmission error. Therefore, we are actually transmitting (0, 0, 1, 1, 1, 0, 0) (1, 1, 1, 0, 0, 0, 0) (1, 1, 0, 1, 1, 0, 0)
a binary string of 7 bits which looks like c = (c1, c2, c3, c4, (1, 0, 1, 1, 0, 1, 0) (0, 1, 1, 1, 0, 0, 1)
7
c5, c6, c7) in the ∧ 2 vector space. Each parity check bit is (1, 1, 1, 1, 1, 1, 1)
set equal to the binary sum of a particular subset of the
message bits. The parity check bits, c5, c6 and c7, can be Since a code space is derived from its parity check
defined according to the following rules [1] equations, the design of the parity check equations
determines the code words in the code space. A good
c5 = c1+ c2 + c4 (1) choice of parity check equations is one that yields a
c6 = c1+ c3 + c4 (2) sufficiently “spaced out” code space, meaning that each
c7 = c2+ c3 + c4 (3) code word is in some sense distant from every other code
word. To correct a mis-transmitted string means to re-
These equations may be restated as assign the string to its correct code word in the code space,
c1 + c2+ c4 + c5 = 0 which is identified by being the closest code word to the
c1 + c3+ c4 + c6 = 0 corrupted string. If two code words are “too close” to each
c2 + c3+ c4 + c7 = 0 other so that they are equally close to a corrupted string,
then we have a 50-50 chance of choosing the correct code
Every code word (source message plus parity check bits) word. Naturally this leads to two questions: how do we
must satisfy the above parity check equations. In matrix measure distance between two code words, and how do we
notation the equations may be expressed as define a well “spaced out” code space such that there is a
unique nearest code word. Hamming distance is the metric
used to define how far apart two strings are in a code space.

inroads – The SIGCSE Bulletin 34 Volume 38, Number 2, 2006 June

 Reviewed Papers

n
For any element x, y ∈ ∧ 2 , the Hamming distance between 2. Code Words in Code Space
x and y, denoted d(x, y), is the number of components
2.1 Encoding Messages
where xi ≠ yi , for i = 1,...,n.
To encode a message means to convert the message from
In our data transmission example, notice that d(x, y) ≥ 4
3 for any (x, y) pair in the code space C7, 4. This means that ∧ 2 to a vector in code space C7,4. We define the converting
any two code words differ by at least 3 bits. In other 4
function F : ∧ 2 → C7,4, as
words, our 16 code words are all at least a Hamming
distance of 3 units apart. This also means that when
F(x1, x2, x3, x4) = F(x1e1 + x2e2 + x3e3 + x4e4)
exactly one bit has been mis-transmitted, the corrupted
= x1g1 + x2g2 + x3g3 + x4g4,
string xp can be corrected to the closest code word with no
ambiguity, since there is only one correct code word xc
where {еi}{i=1,...,4} with the ith component being 1 and all
with d(xp , xc) = 1. When two or more errors occur in 4
transmission, we may possibly detect the existence of others being 0 is the standard basis for ∧ 2 , and {g1, g2, g3,
errors but cannot correct them, because when d(xp , xc) ≥ 2, g4} is a basis for C7,4. Since x1g1 + x2g2 + x3g3 + x4g4= (x1,
there is a strong possibility that the corrupted string is x2, x3, x4) · G, the generator matrix G is also a matrix
actually closer to an incorrect code word. In this case, an induced by the converting function. For example, to
incorrect code word will mistakenly be assigned to correct encode (0, 1, 1, 1), function F converts (0, 1, 1, 1) = e2 + e3
xp. Note that the vector space C7,4 along with the Hamming + e4 to g2 + g3 + g4 = (0, 1, 1, 1, 0, 0, 1) which is the same
distance metric defined on it, give us a metric space in as converting (0, 1, 1, 1) through G by using (0,1,1,1) · G.
which to work. Now the coded message has 3 attached parity check bits.
Since C7, 4 is a vector space, without too much
difficulty, we can compute a basis for C7, 4 (i.e. N.S.(H)) 2.2 Decoding Messages
and place it in row vector form in a matrix, G, as follows: To decode a message means to check the message to
determine if there has been an error, and if so, correct the
1 0 0 0 1 1 0 error and extract the original message. The parity check
0 1 0 0 1 0 1 matrix H will determine if the message is received
G= correctly. Again here we assume there is at most one bit
0 0 1 0 0 1 1
  error. For example when a message is received as
0 0 0 1 1 1 1
0 
This means R.S.(G) = N.S.(H) (i.e. the Row Space of G 1 
equals to the Null Space of H). Since rank(H) = 3, the  
dimension of N.S.(H) = 7 – 3 = 4 = rank(G). The code 1 
 
space C7, 4 generated by G is thus a 4-dimensional subspace x = 1 
7
in the 7-dimensional vector space ∧ 2 . We call G the 0 
 
generator matrix of the code space C7, 4. Since the set of 0 
row vectors, {g1, g2, g3, g4}, of G form a basis for the 1 
 
code space C7, 4, any code word in C7, 4 can be expressed as
a linear combination of gi’s. We also see that the rows of the fact that
the generator matrix G and the rows of parity check matrix
H are orthogonal to each other by observing that is G·HT =
0 
0. Hence N.S.(G) = R.S.(H) = C.S.(HT) (i.e. Column Space
of HT). The column space of H, C.S.(H), provides useful Hx = 0
information regarding the position of the error. For any 0
received string xr, the product of the parity check matrix H
and xr is called the syndrome vector. We see that any indicates x ∈ N.S.(H), which means x was received
3
syndrome vector Hxr ∈ ∧ 2 = C.S.(H) and every non-zero correctly. Therefore, the source message is (0, 1, 1, 1)
3 7 which is x without the attached parity check bits.
vector of ∧ 2 is a column of H. Thus if xr ∈ ∧ 2 but xr ∉
C7,4 (i.e. xr is a string, but not a code word), then Hxr ≠ 0. 2.3 Error Detecting and Correcting
Therefore, in order for xr to be correct, Hxr must be in the If a message is received as
column space of H. We call C.S.(H) the syndrome space
generated by H and xr. We will discuss the details of
syndrome decoding as an alternative decoding method.

inroads – The SIGCSE Bulletin 35 Volume 38, Number 2, 2006 June

 Reviewed Papers

0  Table 1: Parity Check

0  possible bit error
  1 2 3 4
1 failed eqn. 1 x x x
 
y = 0  failed eqn. 2 x x x
1 failed eqn. 3 x x x
 
0  If a string is received as z = (1, 1, 1, 1, 0, 1, 0) and the
0 
  syndrome vector is Hz = (1, 0, 1), then z failed to satisfy
equations 1 and 3. From parity check Table 1, bit 2 was
the fact that not received correctly, and therefore the correct string is the
code word zc = (1, 0, 1, 1, 0, 1, 0), and the correct source
1 message is (1, 0, 1, 1). A syndrome vector of the form (1,
Hy = 1 0, 0), shows failure of equation 1 and therefore bit 5 of the
received string was in error. Table 2 shows how the
1 syndrome vector can be used to find the error bit for an
arbitrary syndrome vector Hv. It is interesting to note that
indicates y was received incorrectly, because y ∉ N.S.(H). if syndrome vector Hv equals the ith column vector in H,
Since the string (0, 0, 1, 1, 1, 0, 0) is the nearest code word then the ith bit of the received string v is in error.
to y in C7,4 (the only code word with distance one from y),
the encoder will correct the received string y to the code Table 2: Error Location Table
word yc = (0, 0, 1, 1, 1, 0, 0). We can see, even by this 0: eqn. satisfied; 1: eqn. failed
simple example, that performing a full-scale nearest eqn. 1 1 1 0 1 1 0 0
distance search through the entire code space is rather eqn. 2 1 0 1 1 0 1 0
tedious, and it should be avoided. A slightly more eqn. 3 0 1 1 1 0 0 1
sophisticated method called “syndrome decoding” can be bit error location 1 2 3 4 5 6 7
used instead.

2.4 Syndrome Decoding

The method of syndrome decoding is used to avoid 3. Conclusion
performing the calculation of Hamming distance between What we have developed in this example is a motivation
xr and every single vector in the code space in order to for the understanding of vector space, basis, null space, row
identify the nearest code word. Rather, the syndrome space, column space and metric space as they apply to data
vector reveals, through the pattern of parity check failures, transmission theory. Hamming code, as it applies to the
the location of the error in xr. A transmitted string y = (0, correction of a single transmission error, is explained via
0, 1, 0, 1, 0, 0) with syndrome vector Hy = (1, 1, 1), vector space theory. Various methods of data transmission
indicates y failed to satisfy all three of equations 1, 2 and 3. and error correction are chosen based on the particular
From Table 1, we see that the syndrome vector indicates an needs of each project by weighing the cost of error
error in bit 4. Correcting the 4th bit in y will correct the correction (reduced speed of transmission) against the need
received string. Hence, the corrected string is the code to correct corrupted data. In order to detect and correct
word yc = (0, 0, 1, 1, 1, 0, 0), and the source message is (0, multiple bit errors, we would have to create a larger and
0, 1, 1). more “spaced out” code space. It is hoped that this
example will show students of applied mathematics and
computer sciences that the study of vector spaces has real
world applications, as well as provide students a solid
example with which they can understand some abstract
vector space concepts.

References
[1] Berlekamp , E., Algebraic Coding Theory, McGraw-Hill Book Company, 1968.
[2] Hill, R., Elementary Linear Algebra with Applications, Saunders College publishing, 1996.
[3] Lin, S., An Introduction to Error-Correcting Codes, Prentice-Hall, 1970.

inroads – The SIGCSE Bulletin 36 Volume 38, Number 2, 2006 June

View publication stats