Coding Concepts and Reed-Solomon Codes

A.J. Han Vinck

University of Duisburg-Essen, Germany

Coding Concepts and
a Reed So
olomon Coddes

Copyrigght 20133: A.J. Han Vinck, Vennlo, the Neth

herlands

All righhts reserved..

ISBN 978-3-98130030-6-3

Publisheer: Institutee for Experim

mental Matthematics, Ellernstr.
E 29,
45326, EEssen, Germ many
Dedication
This book is dedicated to my wife Martina, with whom I shared
almost all of my life. Her support and love made my work possible
and also enjoyable.

I am grateful to my PhD supervisor Piet Schalkwijk for introducing

me to the field of Information theory at the University of Eindhoven,
the Netherlands. The work in his group was the basis for my scientific
career.

I also want to thank the numerous students that participated in my

research during a period of over forty years. During my stay in the
Institute for Experimental Mathematics, also many foreign guests
participated in the research and over 100 papers were published in
journals. Part of these results are presented in this book.

My colleges Hendrik Ferreira, Yuan Luo, Hiro Morita and Kees

Schouhamer Immink, joined me in many research projects and
organization of workshops and conferences. These activities
contributed to the very stimulating research atmosphere in my
working group in the Institute for Experimental Mathematics in Essen.
Contents
Preface

1. Packet Transmission
1.1 Introduction 1
1.2 Transmission Using an Encoding Matrix 3
1.2.1 Packets as symbols of an RS code 3
1.2.2 Binary packets combination 3
1.2.3 Non-binary packets combination 5
1.3 Feedback and Collision Detection 7
1.4 No Feedback, Collision Detection 8
1.5 No Feedback, no Collision Detection 9
1.5.1 Error correction for array-like data structures 9
1.5.2 The access algorithm 13
1.6 Concluding Remarks 15

2. Random Access
2.1 Channel Capacity and Access Models 19
2.1.1 The binary OR channel 21
2.1.2 The M-ary OR channel 23
2.2 Signaling with Random Signatures 27
2.2.1 Signatures with a symmetric input distribution 28
2.2.2 Signatures with an asymmetric input distribution 29
2.3 Signaling with Constructed Signatures 31
2.3.1 The Titlebaum construction 31
2.3.2 Decoding error probability for the Titlebaum 32
construction
2.3.3 Superimposed codes 34
2,4 Signatures for the XOR Channel 40
2.5 Concluding Remarks 41

3. Cooperative Coding
3.1 Introduction 45
3.2 Communication Strategies 46
3.3. Multi User Tandem Communication 53
3.3.1 Point-to-point communication 54
 3.3.2 Communication strategies 55
3.4 Concluding Remarks 61

4. Modulation and Detection

4.1 The Gaussian Channel 64
4.1.1 Error correction 65
4.1.2 Error and erasure decoding 66
4.1.3 Soft decoding 67
4.2 The Combination RS and SPC 72
4.2.1 RS symbols extended with a parity bit 72
4.2.2 Block-wise RS-SPC coding 73
4.2.3 Performance 75
4.3 Non-Gaussian Disturbances 77
4.3.1 M-FSK modulation 77
4.3.2 Detection 78
4.4 Combined Modulation and Coding 82
4.4.1 Permutation codes 82
4.4.2 Permutation codes derived from RS codes 85
4.4.3 Performance for permutation codes 86
4.5 Other Combinations 88
4.5.1 RS-convolutional code combination 89
4.5.2 RS-constrained code 89
4.5.3 RS product code 90
4.6 Concluding Remarks 92

5. The Wiretap Channel

5.1 Wiretap Channel Models 96
5.2 Noiseless Main Channel 100
5.2.1 Single Parity Check code (SPC) 100
5.2.2 Hamming code 101
5.2.3 Reed-Solomon codes 102
5.3 Noisy Main Channel 103
5.4 Noiseless Main Channel and Partly Observed Message 107
5.5 Concluding Remarks 111

6. Biometrics
6.1 Biometric Authentication 115
6.2 Biometric Reconstruction 117
6.3 The Juels-Wattenberg Scheme 120
6.3.1 Juels-Wattenberg Scheme with a biometric of length n 121
6.3.2 Juels-Wattenberg Scheme with a biometric of fixed 122
length t < n
6.4 The Juels-Sudan (JS) Scheme 124
6.5 An Improved Version of the Juels-Sudan Scheme 126
6.6 Concluding Remarks 128
7. Codes with Constraints
7.1 Symbol Avoidance 129
7.2 Constrained Coded Modulation 132
7.2.1 System design 132
7.2.2 Idea for the modulation 133
7.2.3 Idea for the RLL-encoding 134
7.3 Optimum Distance Profile 140
7.4 Same-weight Code Construction 144
7.5 Concluding Remarks 145

8. Memories with Defects

8.1 Defect models 147
8.2 Defect Matching Codes 149
8.2.1 One defect in n positions 149
8.2.2 n-1 defects in n positions 150
8.2.3 Two-defects in n positions 150
8.2.4 A general method for t = pn defects in n positions 153
8.3 Defects and Linear Codes 155
8.3.1 Defect matching 155
8.3.2 Defect matching and random error correction 156
8.4 Defects as Symbols 157
8.5 Concluding Remarks 160

9. Appendix
A.1 A Bit of Information Theory 161
A.2 A Short Introduction to Galois Fields 164
A.2.1 Calculations modulo a prime number, GF(p) 164
A.2.2 Calculations modulo a binary minimal polynomial 165
A.3 Vandermonde’s Determinant 167
A.4 RS Encoding and Syndrome Former 169
A.5 The Decoding Algorithm 175
A.6 Middleton Class-A Impulse Noise Model 178
A.7 Impulse Noise Channel Capacity 181
A.8 Channel Capacity and Water-filling 185
A.8.1 Gaussian channel capacity 185
A.8.2 Capacity of the degraded broadcast channel 186
A.8.3 Water-filling for two parallel channels 187
A.8.4 Water-filling for n parallel channels 188

10. References 190

My co-authors for the referenced papers 197
Index 201
Preface
The material in this book is presented to graduate students in Information-
and Communication theory. The idea is that we give an introduction to
particular applications of information theory and coding in digital
communications. The goal is to bring understanding of the underlying
concepts, both in theory as well as in practice. We mainly concentrate on our
own research results. After showing obtainable performance, we give a
specific implementation using Reed-Solomon (RS) codes. The reason for
using RS codes is that they can be seen as optimal codes with maximum
obtainable minimum distance. Furthermore, the structure of RS codes
enables specific applications that fit perfectly into the developed concepts.
We do not intend to develop the theory of error correcting codes. We
summarize our idea in Figure 0.1.

concept development

performance estimation

practical embodiment

application of
RS codes

performance evaluation

Figure 0.1 Concept of the book

i
Chapter 1 considers networks where information is transmitted in packets.
Reliable packet transmission can be realized via networking protocols using
feedback and error control coding. When the data in the packets are code
words of an RS code and in addition the packets have an identification (ID)
number, provided by the network protocol in use, we can use error control to
improve the throughput efficiency especially in the situation where packets
get lost. Lost packets can be recovered by erasure decoding.

The random access situation, where users transmit packets without

knowledge about other users is solved using the Aloha protocol. We discuss
different random access strategies. The first one is the classical Aloha
protocol, where feedback is given to the users about successful or not
successful transmission (collision). The theoretical throughput of the Aloha
system η = Ge-G, where G is the total offered load by the users involved in
the random access protocol, in packets per time slot, see Figure 2.

e-1

0
0 1 G
Figure 0.2 The throughput for the slotted Aloha

We discuss a randomized scheme using RS codes, where there is no

feedback, but collision detection. We also present a simple and efficient
error correction scheme for array-like data structures. This scheme is used
for packet transmission using the Aloha protocol without feedback and
without collision detection.

In Chapter 2 we develop the concepts of random access communications,

where we are especially interested in the situation where users cannot use
knowledge of the participants in the access situation. Information theory
considers the capacity or maximum mutual information of multi access

ii
channels (MAC) for users that are permanently sending information. In
random access the users are not permanently active, and in addition cannot
communicate with each other. We discuss coding in a multi user concept
where the set of active users is small compared to the total amount of
potential users. In this case, time - or frequency division is an inefficient
way of communication. We introduce three models for “non-trivial” access.
For these models we calculate the maximum throughput and the way how to
obtain this maximum throughput. We consider transmission using
signatures. For a simple construction we can calculate the performance and
compare the results with random signatures. We give a class of codes that
can be used to uniquely determine the set of active users from the composite
signature at the channel output.

Interesting applications for RS codes appear in Chapter 3, for channels that

are not typical. One of the non-typical channels is the power line
communications channel. In this channel we have all different types of noise
and disturbances. RS codes give interesting optimal solutions to some of the
problems.

One of the dominant parameters in power line communications is that of

attenuation. Attenuation can vary between 10-100 dB per kilometer. In
power line communications, attenuation prevents signals to propagate over
distances that are longer than say 500 meters. Investigations to overcome the
high attenuation are therefore highly important. We discuss the efficiency of
communication over links that are in tandem. This situation occurs when
users in a power line communication network are connected to the same line
and the power lines can be seen as “bus systems”, where all connected users
can listen to the signals present on the bus. There is an interesting connection
between links in tandem and the classical problem of broadcasting in
information theory. We relate the tandem channel with the degraded
broadcast channel and give the information theoretical capacities.

A popular topic in coding is the application of soft decision decoding for

additive white Gaussian noise channels. The reason for this is the
performance improvements in the decoding error rate that can be obtained. It
is well know that, due to complexity reasons, RS codes cannot be decoded
using soft decision information. In Chapter 4, we first explain the idea of soft
decision and then give examples of the applications. We combine RS codes
with single parity check codes and show that a 3 dB gain can be attained at
low complexity.
In the second part of the chapter we consider a special class of codes,
permutation codes. These codes are effective against different kinds of noise,

iii
like impulse- and narrowband noise. Extended RS codes with low rate (2/n)
can be seen as an optimum class of permutation codes. We develop the
concept of same-weight codes to be used for channels with permanent
disturbances.

In Chapter 5, we describe the classical problem of data transmission in the

presence of a wiretapper listening to the communication between two
parties. The encoder’s objective is to maximize the amount of information
transmitted to the legal receiver, while keeping the wiretapper as uninformed
as possible. We consider a noiseless - and a noisy main channel, see Figure
0.1 and Figure 0.2, respectively.

key k key k

cryptogram c
encrypt decrypt
e(m,k) = c d(c,k) = m

message m message m

Figure 0.1 Schematic representation of the noiseless Shannon

cipher model.

key k key k

noise
cryptogram c
encrypt decrypt
e(m,k) = c d(c, k) = m

message m message m

Figure 0.2 Schematic representation of the noisy cipher system

iv
For both situations, the use of a special form of RS encoding gives
optimum results.

In Chapter 6, we show that one of the interesting consequences of the

developed theory is that there is a close connection with the problem of
secure biometric authentication and verification. The biometric verification
scheme as developed by Juels-Wattenberg can be seen as a noisy cipher
system. The biometric properties act as a key at encryption, and as a noisy
key at decryption.

enrollment en noise verification

bio bn bio bn

cn + bn data base
n
c cn + en
encoder decoder

? X’
random data X  Hash(X)  =  Hash(X’)

Figure 0.2 Schematic representation of the Juels-Wattenberg scheme

We include the Juels-Sudan scheme and show that there is an interesting

implementation of both schemes using RS codes. The chapter includes
performance bounds for the False Acceptance Rate and the False Rejection
Rate.

Chapter 7 contains topics that can be seen as constrained coding. The first
topic is the avoidance of certain symbols to occur in the output of an RS
encoder. The second topic is the combination of RS and run-length
constrained coding with application in bandwidth limited channels. In
communication systems, coding is a fixed part of the design. To change a
code is impossible due to standards and already implemented decoding
algorithms. In cognitive systems, we want to adapt the efficiency of a
transmission scheme to the actual circumstances. For this we have to be able
to change the modulation and also the error correcting - or error detecting

v
code. Some concepts are well known, such as puncturing or
shortening/lengthening an encoding matrix of a linear code. In this way, the
efficiency of the code is changed by changing the length n of the code
words. We choose another method: row extension/deletion of the encoding
matrix for an (n,k) code, and thus change the parameter k.

An important part of a computing system is the memory. Due to

improvements in process technology and clever circuit design, we can
produce large memory systems on a chip with a high packing density. A
high packing density has its limits and may cause errors in the memory cell
structure. A popular error model is that of defects. A defect always produces
the same output when being read, irrespective of the input. The model for
defective memory cells is given in Figure 0.3.

0 0 0 0 0 0

1 1 1 1 1 1
no defect defect-0 defect-1

Figure 0.3 Model for the defects in a memory

Kuznetsov and Tsybakov showed that, when the writer knows and the reader
does not know the value and the position of the defect, the writing capacity
is (1-p) bits/memory cell. In Chapter 8, we show how to use RS codes as
optimum symbol defect matching codes.

The appendix contains the background information about RS codes as

needed for the understanding of the rest of the chapters where we discuss the
different applications. We also give some basic properties of the concepts of
entropy in information theory. We give a brief “engineering” summary of
the theory for Galois fields as far as we need this. We include the necessary
properties of the Vandermonde determinant and we complement the
description of RS codes with a possible decoding procedure. In the last parts
we explain the idea of water-filling for two and more parallel channels.

vi
vii
viii
 Chapter 1-Packet Transmission

Chapter 1

Packet Transmission
In networks, data is transmitted in the form of packets. A packet consists of a
block of data bits plus control information that helps in its proper routing and
reassembly into the original form at the receiving end. We assume that
packets have an identification number (ID) provided by the network protocol
in use. Reliable packet transmission can be realized via networking protocols
using feedback and error control coding. Error control coding can improve
the throughput or transmission efficiency especially in the situation where
packets get lost.

We discuss different protocols depending on the actual networking situation:

- the classical Aloha protocol [1], where feedback is given to the users
about successful or not successful transmission;
- a protocol with randomized packet transmission using Reed-Solomon
(RS) codes, when there is no feedback but there is collision detection;
- a simple and efficient error correction scheme for array-like data
structures. This scheme is used for packet transmission using the Aloha
protocol without feedback and without collision detection.

1.1 Introduction
RS codes play an important role in networking. We will highlight some of
the “simple” applications. We first recall the main properties of linear error
correcting codes that are needed for the protocol descriptions.

1
 Chapter 1-Packet Transmission

Property 1.1 The code words of a linear (n,k) error correcting code are
linear combinations of the rows of a k × n encoding matrix Gk,n of rank k, i.e.
cn = xkGk,n, where xk and cn are of length k and n, respectively.

We may distinguish between bit- and symbol oriented codes. For symbol
oriented codes over GF(2m), the components of xk, cn and Gk,n are m bits
wide. The minimum number of differences (measured in symbols) between
any two code words is denoted as dmin.

Property 1.2 For an (n,k) linear error correcting code, the minimum distance
dmin is upper bounded by dmin ≤ n – k + 1. RS codes achieve the upper bound
with equality.

Property 1.3 An (n,k) error correcting code with minimum distance dmin is
able to detect at least t symbol errors for dmin ≥ t +1.

Property 1.4 An (n,k) error correcting code with minimum distance dmin is
able to correct at least t symbol errors for dmin ≥ 2t +1.

If the position of an error in a code word is known, without knowing the

value of the error, we call this an erasure.

Property 1.5 For an (n,k) linear code with minimum distance dmin, dmin-1
erasures can be corrected.

Property 1.6 As a consequence of property 1.5, any k × (n - (dmin -1)) sub-

matrix of the encoding matrix Gk,n has rank k.

A non-zero input gives a non-zero output, even if we delete dmin - 1 columns

of the encoder matrix. This property plays an important role in the recovery
of erased or missing packets.

Error-detection is a powerful tool to decrease the decoding error rate. The

idea is:

- use a linear (n,k) block code to transmit information;

- if the received word is not equal to one of the possible transmitted code
words, errors are detected.

Misdetection occurs iff (if and only if) the error pattern changes a
transmitted code word into another valid code word. For linear codes, the
undetected error probability is determined by the probability that we do

2
 Chapter 1-Packet Transmission

receive a code word with dmin differences from the transmitted code word. In
[4] we discuss the undetected error probability for linear block codes on
channels with memory.

For narrow band applications, packet transmission combined with Automatic

Repeat reQuest (ARQ) is a very robust way of communicating in a point-to-
point connection. For this, data is encoded with an (n,k) linear error
correcting code with code efficiency R = k/n < 1. Packets detected to be in
error, are requested to be retransmitted. In general, for a packet error rate p
and code efficiency R, the overall transmission efficiency for the basic ARQ
is

ηARQ = (1-p) R.

Since long packets are certainly received in error, the packet length needs to
be adapted to the channel error behavior. This requires accurate modeling.

A problem we do not touch is that of packet synchronization. It can be

shown that there is a synchronization problem when there is a gap in time
between the transmitted packets, see [5,6].

1.2 Transmission Using an Encoding Matrix

1.2.1 Packets as symbols of an RS code

We consider the transmission of n packets, where every packet contains a

symbol (typically 8-16 bits) from an RS code word. Since the minimum
distance of the RS code is n – k + 1, we can accept n - k erasures or we need
k correctly received packets to be able to reconstruct (decode) the encoded
information.

1.2.2 Binary packet combination

An alternative option is to encode k binary data words (P1, P2, , Pk) with a
binary encoding matrix Gk,n for a code with minimum distance dmin. The ith
data word Qi, 1  i  n, is encoded as

Qi = g1,i P1  g2,i P2    gk,i Pk,

3
 Chapter 1-Packet Transmission

where gj,i is the binary component from Gk,n at row j and column i.

Using Property 1.6, any sub-matrix with dimension k × (n - (dmin -1)) has
rank k. Thus, if at the receiver we do receive at least dmin -1 correct packets,
we are able to decode the transmitted k data words by using the ID of the
lost (erased) packets.

Example We use a (7,3) encoding matrix for a code with minimum

distance 4, i.e.

1000111
G3,7 = 0101011 (1.1)
0011101

Any sub-matrix with dimension k × (n - (dmin -1)) = 3 x 4 has rank 3 and can
thus be used to reconstruct the three encoded data words when not more than
3 transmitted packets out-of -7 are lost (erased).

Suppose that we use matrix (1.1) for the encoding of data words that are RS
code words. The RS codes are good error detection codes and thus after
transmission, the receiver knows whether a data word contains errors or not.
The data words detected in error can be declared to be erased or missing.
The same decoding principle applies as before.

Example We encode three RS code words over GF(2m) as

(RS1, RS2, RS3) G3,7 = (Q1, Q2, ···, Q7).

For this encoding scheme, Q7 = (RS1 ⊕ RS2 ⊕ RS3), where ⊕ denotes the
exclusive-OR (XOR) of the code words RS1, RS2 and RS3. By linearity, Q7 is
also a code word and can thus be treated as a code word from the RS code. If
Q3, Q4 and Q5 are lost, we can use the columns 1, 2, 6 and 7 to reconstruct
(RS1, RS2, RS3). The part of the encoding matrix

1011
G3,4 = 0111 ,
0001

has rank three and the 3 x 3 matrix formed by columns 1, 2 and 4 has an
inverse. Multiplying the received vector of data words (Q1, Q2, Q7) with Ginv,
we find back RS1, RS2 and RS3, i.e.

4
 Chapter 1-Packet Transmission

101
(Q1, Q2, Q7) · 0 1 1 = (RS1, RS2, RS3).
001

Another option for the receiver is to decode the received RS code words.
After successful decoding of dmin - 1 received code words, we can do the
inverse operation and reconstruct the three RS code words.

1.2.3 Non-binary packet combination

Suppose that we have k data words, each with symbols from GF(2m). We
combine these data words using an encoding matrix for an RS code, again
with symbols from GF(2m). Since the data words are multiplied with
elements from GF(2m), the linear combinations are again data words with
symbols from GF(2m). If the combining matrix has minimum distance dmin,
we can reconstruct a maximum of dmin - 1 erased (or missing) data words.

For an erasure rate of pn packets, (n - k) = pn redundant packets allow for

correct decoding of the erased (missing) data words. The transmission
efficiency of this scheme is thus

k/n = ηcoded = (1- p),

which is roughly the same as for ARQ. The disadvantage is that we have to
wait for k correctly received packets before decoding can start. The
advantage is that we do not need the feedback.

Note This method can also be seen as a kind of parallel processing. Instead
of working on packets, we could also work on a symbol level. If we consider
only the ith symbol from each packet, we have the normal RS encoding, see
1.2.1. In the present situation, the “side-information” is available for all
symbols in a packet and not for each symbol individually.

5
 Chapter 1-Packet Transmission

Example Suppose that we have k = n - 2 data words. These words are

encoded with the RS encoding matrix Gn-2,n as

(P1, P2, ···, Pn-2) • 100 0 αn-2 1 = (C1, C2,···, Cn).

010 0 αn-1 1
··· ···
0 0 ··· 0 1 0 α2 1
0 0 ··· 0 1 α 1

The RS code has minimum distance 3 and thus 2 erasures or missing data
words can be corrected. Using the syndrome former,

αn-2 1
αn-1 1
HT= ···
α 1
1 0
0 1

one can also correct one data word that is in error. Suppose that we have one
data word in error and that the error word is E. Then,

(C1, C2, ···, Ci  E, Ci+1, ···, Cn) HT = (S1, S2).

The syndrome (S1, S2) can have the following output:

(0, E) error E in Cn;

(E, 0) error E in Cn-1;
(αi E, E) error E in position n – i - 1, 1 ≤ i ≤ n - 2.

From the correct (C1, C2, ···, Cn) the words (P1, P2, ···, Pn-2) can be
reconstructed.

Note We can use RS code words as input words and since then the output
words are also RS code words, these can be decoded first. A decoding error
can then be corrected with the above method in a following step.

6
 Chapter 1-Packet Transmission

1.3 Feedback and Collision Detection

One of the famous random access transmission protocols is the Aloha
protocol [1]. We consider only a simplified version of the slotted Aloha
protocol. Transmitters have an input buffer, where incoming packets are
stored. In the simplified slotted Aloha protocol with T active users, we
assume the following steps:

1. an active user takes a packet from the input buffer with probability p. A
packet is transmitted in the next available time slot. If no packet is taken
from the buffer, with probability (1-p), we do not use the next time slot
and repeat the procedure;
2. if a collision (two or more users use the same time slot) is detected and
reported back to the transmitter, via a noiseless feedback, the transmitter
repeats the last packet in the next time slot with probability p;
3. when no collision is detected, we go to step 1.

The transmission efficiency for this Aloha protocol is given by the fraction
of successful packet transmissions (no collision), i.e.

η = pT( 1 – p )T-1, (1.2)

which approaches 1/e packet per time slot for T = 1/p. For our model, we can
define G = pT as the total average traffic offered to the protocol by the users.
We then have for small p

η ~ Ge-G , (1.3)

which is illustrated in Figure 1.1.

e-1

0
0 1 G
Figure 1.1 The transmission efficiency for the slotted Aloha

7
 Chapter 1-Packet Transmission

Complicated problems occur in the calculations, when the model is not as

ideal as in our example.

1.4 No Feedback, Collision Detection

RS codes can be used in a collision detection protocol. It works as follows.
Suppose that T ≤ Z active users from a large population transmit symbols
from an RS code of length n using a transmission array with Z rows and n
columns. Every user has a particular “random” signature of length n that
indicates in which row and column a symbol can be transmitted. We denote
nPe as the number of expected collisions. For a particular user, a collision
occurs if at least one of the remaining T-1 users uses the same slot, i.e. the
number of expected collisions

1 T 1
nPe  n(1  (1  ) ). (1.4)
Z

For nPe ≤ n-k, we expect correct decoding at the receiver. From (1.4) we
have the condition that for correct decoding

T -1
k 1 -
 (1  ) T 1  e Z .
n Z

The overall transmission efficiency of the protocol, for T users each sending
k information symbols in an array with dimension Z × n, is then

T -1
Tk T - Z
η  e . (1.5)
Zn Z

For large Z, The maximum approaches e-1 for T/Z → 1, which is the same as
for the slotted Aloha with collision feedback.

Example In Figures 1.2, 1.3 and 1.4 we give the code array for four users
with code words of length 6 symbols, the “spreading” code array with 6
rows and 6 columns, and the received “de-spread” code array, respectively.

8
 Chapter 1-Packet Transmission

c11 c12 c13 c14 c15 c16

c21 c22 c23 c24 c25 c26
c31 c32 c33 c34 c35 c36
c41 c42 c43 c44 c45 c46

Figure 1.2 Code array for 4 users, code words are horizontal

c11 c24 c46

c22
c12 c34 c35
c21 x x x
c41 c32 x c16
c31 c42 c43 c25 c26

Figure 1.3 “Spreading” array, where x indicates collision

c11 c12 x x x c16

c21 c22 x c24 c25 c26
c31 c32 x c34 c35 x
c41 c42 c43 x x c46

Figure 1.4 “De-spread” code array for the active users

1.5 No Feedback, no Collision Detection

1.5.1 Error correction for array-like data structures

Before we describe the protocol, we have to introduce error correction for

array-like data structures, see also [2,3,7].

9
 Chapter 1-Packet Transmission

A packet array Pk,N consists of N columns and k rows, as shown in Figure

1.5a. Every column in this array belongs to a specific information word to be
encoded. The elements of this array are symbols of GF(2m).

P1 P2 ••• PN
p11 p12 ••• p1N row 1
p21 p22 ••• p2N row 2
Pk,N =
••• ••• •••

pk1 pk2 ••• pkN row k

Figure 1.5a Packet array Pk,N with k rows and N columns

A code array Cn,N with n rows and N columns is given in Figure 1.5b. The
elements of this array are symbols of GF(2m). Every column contains the n
symbols of an RS code word.

C1 C2 ••• CN
c11 c12 ••• c1N row 1
c21 c22 ••• c2N row 2
Cn,N =
••• ••• •••

cn1 cn2 ••• cnN row n

Figure 1.5b Code array Cn,N with n rows and N columns

The encoding of an array of information is given in Figure 1.6. The

information array, consisting of N columns of length k is multiplied by the
encoding matrix GT leading to a code array of dimensions n × N. The
arrangement of code words as columns of an array can be interpreted as
block-interleaving.

Transmission of the array Cn,N is done row wise, symbol by symbol. We

furthermore assume that channel output is binary (hard decision). The errors

10
 Chapter 1-Packet Transmission

inserted by the channel can be described by an additive error array En,N,

where Rn,N = Cn,N  En,N. The channel is assumed:

- to behave such that each row of a received array is either error-free or

corrupted by many symbol errors;
- the errors which disturb the rows of the array Cn,N are such that the rows
of En,N are linearly independent;
- the number of erroneous rows is t.

k N

=
n GT n

Encoding matrix packet array code array Cn,N

Figure 1.6 Encoding operations on a block of information

At the receiver side, we can use the property of the check matrix Hn-k,n, and
calculate the syndrome array

Sn-k,N = Hn-k,n Rn,N

= Hn-k,n (Cn,N  E n,N)

= Hn-k,n E n,N.

We visualize the process as given in Figure 1.7

In order to use the properties of Sn-k,N, we will take a closer look at the
syndrome array. The t rows that are in error determine the sub-array Esub and
the collection of row numbers form the set E. Consequently, the numbers in
E define t columns in Hn-k,n, called the sub-array Hsub and clearly,

11
 Chapter 1-Packet Transmission

Sn-k,N = HsubEsub.

We now will consider the performance of the scheme. The assumption is that
t independent error words of length N corrupt the code array Cn,N. For t ≤ ( n
- k), t columns of Hn-k,n are linearly independent and hence rank(Hsub) = t.
Furthermore, rank(Esub) = t because linearly independent error words are
assumed. It thus follows that

rank(Sn-k,N) = rank(Hsub · Esub) = t.

For t < (n-k) the syndrome array Sn-k,N does not have maximum rank.
Consequently, (n-k-t) all-0 rows can be constructed from Sn-k,N by
performing elementary row operations according to Gaussian elimination.
Let the resulting array with (n-k-t) all-0 rows be denoted by S0. Performing
on Hn-k,n the same row operations that were performed on Sn-k,N we get the
matrix H0 where H0 Rn,N = S0.

N N

n-k Hn-k,n E n,N = Hn-k,n E n,N

Figure 1.7 Array syndrome forming

The reconstruction of the original information follows immediately. We use

the following observations:

- we need k correct positions in a code word to be able to reconstruct the

original code word;
- an RS code with minimum code word weight k+1 is generated by Hn-k,n;

12
 Chapter 1-Packet Transmission

- the product H0 × En,N = 0n-k-t, N, can only be true if the non-zero symbols
of H0 correspond to error free rows of En,N.

Hence, H0 indicate at least k+1 error free positions in Cn,N and thus the
original encoded information can be reconstructed. For RS codes, we need
only one row from H0. For general linear codes, we refer to [2].

Remark We want to state that the main ideas of the presented algorithm
were presented in a publication of J. J. Metzner and E. J. Kapturowski [7].

1.5.2 The access algorithm

The Aloha system is a simple random access scheme, but it requires collision
detection and feedback from the receiver or channel to the transmitters. We
will now use the proposed decoding algorithm to construct a random access
scheme that requires no feedback. No collision detection is required because
our decoding algorithm performs this collision detection inherently by
searching for error-free rows. Consequently, we do not need redundancy in
the rows for error detection. In conclusion, collisions are not detected as
such, but appear as errors. Therefore, normal erasure decoding is not
possible.

C1 C2 C3 C4 •••
c11 c12 row 1
c21 c22 row 2
c31 c32
c41 c42
c51 c52 row 5

Figure 1.8 Code array for a particular user

Assume that we have T independent users that have access to Z parallel and
independent transmission channels. Each user encodes his data as described
in Figure 1.8. Let C be an RS code with code symbols from GF(2m). In
addition, every user possesses a signature sequence (s(1), s(2), ···, s(n)), of
integers s(i), with 1  s(i)  Z . They are chosen with

13
 Chapter 1-Packet Transmission

Probability(s(i) = j, 1  j  Z) = 1/Z.

The signature connected with each user is known at the receiver side. At
time interval i the transmitter selects channel s(i) for the transmission of the
ith code row. Furthermore, the users are assumed to be row-synchronized. If
two or more users use the same channel at the same time, we assume that the
received row is erroneous. Hence, we have exactly the same situation as is
required for the proposed decoding algorithm.

r11 r42
r22
r21 r43 r53
r12 x x
r14 r23 x
r13 r24 r34 r52

Figure 1.9 Randomized transmission (6 rows, 5 columns), where x

indicates collision, T = 4 active users, Z = 6 parallel channels

C1 C2 C3 C4 •••
c11 c12 c13 c14 row 1
c21 c22 c23 c24 row 2
x x x c34
x c42 c43 x
x c52 c53 x row 5

Figure 1.10 Received code array for a particular user

A user manages to decode the received array correctly if not more than (n-k-
1) code rows are corrupted by the other users and if the error vectors that
result from the collisions are linearly independent. Whether the second
condition is fulfilled depends on how the channel reacts in the case of a
collision. We consider the case where a collision as channel output is a
random sequence. Hence, all error vectors are equally likely.

The probability that a row is received as a collision is given by

14
 Chapter 1-Packet Transmission

T 1
 Z 1 
Pe  1    .
 Z 

We can expect correct decoding with an arbitrary small decoding error

probability for large block length n and constant rate k/n if

nPe < (dmin – 2) = (n – k - 1),

or

Pe < 1 – k/n.

The transmission of one code array per user requires a total of Z×n×N
symbol slots. The total transmitted un-coded information is T×k×N =
T×n×N×r symbols. Substituting Pe = 1- k/n, we get the following expression
for the transmission efficiency of this random access scheme

T 1
T  Z 1
η   .
Z Z 

For large Z we get

T
  Ge  G , with G  .
Z

The efficiency of the proposed system is the same as the efficiency of the
Aloha protocol. The maximum transmission efficiency is equal to e-1.

1.6 Concluding remarks

The classical way of handling corrupted packets in a point-to-point
connection in a network is that of error detection and retransmission. We call
this method ARQ. For an error rate of p packets per time slot, the
transmission efficiency approaches (1-p)R, where R is the loss in code
efficiency due to the needed redundancy. We can also apply ARQ for
missing packets with a more complicated protocol.

Packets can be transmitted in combination with an error correcting code. The

RS encoding can be used with a redundancy equal to the amount of lost,

15
 Chapter 1-Packet Transmission

erased or collided packets. We can use the erasure option in the decoding
process when missing or erroneous packets can be identified. Furthermore,
feedback is not necessary.

The Aloha protocol is a basic random access protocol in networking when

feedback and collision detection is available. This probabilistic repeat
request protocol has a maximum transmission efficiency of e-1 packets per
packet slot. A simple interleaved RS coding scheme can obtain the same
performance without feedback. We describe a protocol that corrects packet
errors without feedback and collision detection. The protocol was part of a
PhD thesis by Christoph Haslach1. We acknowledge the fact that, in
principle, the protocol was described by Metzner and Kapturowski [7].

The packet reconstruction methods are summarized in Figure 1.11.

ARQ + feedback
+ erasure information

block encoding - feedback

+ erasure information

Aloha random access + feedback

+ erasure information

coded interleaving - feedback

+ erasure information

coded interleaving - feedback

- erasure information

Figure 1.11 Overview of described communication protocols

16
 Chapter 1-Packet Transmission

A problem, not considered here, is that of error detection. Packet

transmission protocols very often depend on the availability of error
detection via a Cyclic Redundancy Check (CRC). A report on this topic can
be found in Kuznetsov et al.2

Packets usually contain a header (prefix) for synchronization purposes. Jim

Massey3 derived the conditions for the best prefix for continuous
transmission of packets. However, when there is a time gap between packets,
called burst mode, this prefix has to be changed. A new optimum design rule
is therefore necessary4. This topic was part of the PhD thesis of Adriaan van
Wijngaarden5. When markers are used in the header, these markers are not
allowed as a pattern in the packet itself. This reduces the transmission
efficiency and optimum markers have to be found. Results were obtained in
a cooperation with Hiro Morita6 . In [50] we consider a symbol oriented
method for synchronization to be combined with RS codes.
1
Christoph Haslach, Array-Codes auf der Basis Interleavter Block codes, Universität Essen,
1999
2
A. Kuznetsov, F. Swarts, A.J. Han Vinck and H.C. Ferreira, "On the Undetected Error
Probability of Linear Block Codes on Channels with Memory," IEEE Trans. on Information
theory, Vol. 42, No. 1, pp. 303-309, Jan. 1996
3
J. L. Massey. "Optimum frame synchronization ". IEEE trans. comm., com-20(2):115-119,
April 1972
4
A.J. Han Vinck and A.J. van Wijngaarden, “On Synchronization for Burst Transmission,”
IEICE, pp. 2130-2135, Nov. 1997
5
Adriaan van Wijngaarden, Frame Synchronization Techniques, Universität Essen, 1998
6
Hiroyoshi Morita, Adriaan J. van Wijngaarden and A.J. Han Vinck, ”On the Construction
of Maximal Prefix-Synchronized Codes,” IEEE Trans. on Information Theory, pp. 2158-
2166, Nov. 1996

17
Chapter 1-Packet Transmission

18
 Chapter 2-Random Access

Chapter 2

Random Access
We discuss coding in a multi user concept where the set of active users is
small compared to the total amount of potential users. The active
transmitters use signatures to be transmitted over a common channel. We
introduce the concept of q-ary superimposed codes, and we give a class of
codes that can be used to uniquely determine the set of active users from the
composite signature at the channel output. The so called T-user M-frequency
noiseless multiple access channel without intensity information, as defined
by Chang and Wolf [8], is an example of such a channel.

Information theory considers the capacity or maximum mutual information

of multiple access channels (MAC) for users that are permanently sending
information. In random access the users are not permanently active, and in
addition do not cooperate with each other. As a reference, we recommend
[96].

2.1 Channel Capacity and Access Models

In Information theory we model random access communications using
simple channel models. In Figure 2.1 we give three models for “non-trivial”
binary multiple access channels, the OR, the XOR and the θ channel,
respectively. We assume that a user has as input the symbol 0 or 1, and the
channel input is 0 in case a user is not active.

19
 Chapter 2-Random Access

OR channel: Y = X1.OR.X2.OR.  .OR.XT

XOR channel: Y = X1.XOR.X2.XOR.  .XOR.XT

θ channel: Y = 1 if more than θ inputs are 1, otherwise Y = 0

Figure 2.1 Three different binary multiple access models

Uncoordinated access
In uncoordinated multiple access one assumes that T individual users, out of
a large community of U users, can access the communications channel
without knowledge of the identity of the other users. Furthermore, the
number of active users T is small compared with the total number of
potential users in the network. The users are time aligned, i.e. have the same
clock. The decoder for a particular user does not use the knowledge it might
have about the other users and also considers the influence of the other users
as interference or noise. The general model is given in Figure 2.2.

X1 R1
X2 Y R2
 MAC 
XT RT

Figure 2.2 The T-user random multiple access channel model

To calculate an achievable rate (transmission efficiency or throughput at a

vanishing error probability) in a random access situation, we assume that
every user has the same input probability distribution. Furthermore, we
consider the problem as a “normal” channel coding problem and thus the
achievable sum rate for the random access channel is equal to the sum of the
individual achievable rates, i.e.

R uncoordinated  T max I(Xi ; Y) . (2.1)

P(X i )

The maximum achievable rate is also called capacity.

20
 Chapter 2-Random Access

Coordinated access
In the situation where the users are coordinated, i.e. know each other’s
identity, the decoders can use the information simultaneously transmitted by
all users. The capacity region [22] is given by the closure of the convex hull
of all rate pairs R1 , R2, , RT satisfying

Ri  I(Y;Xi | all Xj, Xj  Xi), (2.2a)

R1 + R2 +  + RT  I(Y ; X1, X2,  , XT), (2.2b)

for a probability product distribution f(X1, X2, , XT) = f(X1)f(X2)f(XT).

In a fully coordinated situation, the number of available time slots or carrier

frequencies (dimensions) determine the number of users and all dimensions
are used by the participants. Examples of this method are Time Division
Multiple Access (TDMA) and Frequency Division Multiple Access
(FDMA). The transmission efficiency is optimal if all users do indeed use
the communication channel. In all other situations, the channel contains idle
or not-used dimensions. The principle drawback of dimension division is
that each of the U potential users gets only 1/Uth of the dimensions. The
channel contains idle or not-used dimensions when less than U users are
active. Furthermore, the DMA schemes suffer from inflexibility inherent in
pre-assigned dimensions. To accommodate more users than the available
number of dimensions, a demand assignment protocol must be established.

2.1.1 The binary OR channel

The channel mostly investigated in literature is the binary OR access

channel. The binary-input/binary-output relation for this channel is given by

Y = X1.OR.X2.OR..OR.XT. (2.3)

For the binary input T-user OR-channel, the output Y is 1 if at least one
input is equal to 1. If all inputs are 0, the output is also 0. This model can be
seen as a model for optical pulse modulation. The equivalent “AND“
channel can be found in communication busses like the “Wired-AND”
computer connection or the CAN (Controller Area Network) network. In

21
 Chapter 2-Random Access

Figure 2.3 we give the transmission channel as it is seen by a particular user

for p0 = 1 – p1 = p.

pT-1
p0 = p 0 0

Xi 1-pT-1 Y

p1= 1-p 1 1
1

Figure 2.3 The binary OR channel model for user i

Under the specified conditions, the achievable rate (2.1) for the
uncoordinated OR channel becomes

R OR  T max I(X i ; Y)
P(X i )
(2.4)
T T 1
 T max{h(p )  ph(p )},
p

where h(p) = - plog2p - (1- p)log2(1- p) is the binary entropy function. For
(1 - p) = 1  ln2, we obtain that for large T the sum rate is lower bounded by
T

ROR, T  ln2 = 0.6931 bits/transmission. (2.5)

Numerically, one can show that (2.4) approaches (2.5) from above and we
postulate that (2.5) is indeed the limiting capacity for the uncoordinated OR
channel [9].

Evaluation of (2.2) in the coordinated case, gives a maximum sum rate of 1

bit/transmission. This rate can easily be obtained by using a time-sharing
transmission strategy, where every user gets a unique assigned time slot.
Hence, the rate loss due to the uncoordinated access is about 30%.

22
 Chapter 2-Random Access

2.1.2 The M-ary OR channel

The M-ary OR channel has M-ary inputs and at each time instant the
detector outputs which subset of symbols occurred as input to the channel at
that time instant, but not how many of each occurred. For M symbols, we
can thus have 2M – 1 different sets of outputs.

Coordinated access
Chang and Wolf [8] showed that capacity of the coordinated M-ary OR
channel approaches (M-1) bits per transmission. This capacity can be
obtained by a very simple time-sharing strategy. Assume that we divide the
T active users into groups of size (M-1). Within one group the users are
numbered from 1 up to M-1. User i, 1  i  M-1, uses as the transmitting
symbols 0 and i. At the receiver, we are able to detect whether user i uses
position 0 or i and thus, for M-1 users we transmit M-1 bits per transmission.
Note that here we have a “central“ symbol 0.

0 0

0,1

0,2
in out
0,1,2

1 1

1,2

2 2

Figure 2.4 The 3-ary OR channel model

Example For M = 3, T/2 users use the pair (0 or 1) as channel input, the
other T/2 users use as input (0 or 2). The output of the channel is thus (0),
(0,1), (0, 2), or (1, 2). Note that in this coordinated situation the input of the
channel is uniquely decodable.

23
 Chapter 2-Random Access

Uncoordinated access
Suppose that for the uncoordinated M-ary OR channel, some coordination
between the T users exists. Namely, a fraction T/(M-1) of users uses the
input symbol pair (0,i), 1  i  M - 1. According to the previous section,
transmission per group can be considered as transmission over a Z-channel,
where one output contains the symbol i and the other outputs contain only
symbols not equal to i. The achievable sum rate for this channel is ln2 bits
per transmission. Hence, for M-1 groups we obtain a normalized sum rate of
(1 - 1/M ) ln2 bits per dimension.

Example For the example M = 3, as given in Figure 2.4, we have 7 subsets

at the output: {0}, {1}, {2}, {0,1}, {0,2}, {0,1,2}, and {1,2}. The input
probability distribution is chosen as: P(0) = 1 - 2 ln2/T, and P(1) = P(2) =
ln2/T. The limiting input distribution shows that for one symbol the
probability approaches 1. The asymptotic rate R(3,T) approaches 2 ln2 bits
per channel use, or a total of 2/3 ln2 bits per dimension.

For M > 3, we postulate a probability distribution. For this distribution we

calculate the maximum mutual information which is, from an information
theoretical point of view, equivalent to the achievable rate or -error rate.
This result is thus a lower bound for the capacity, since channel capacity is
the maximum mutual information over all possible input probability
distributions. We give an achievable rate for the uncoordinated M-ary OR
channel in Theorem 2.1.1. The achievable rate reduces from (M-1) bits per
channel use in the fully coordinated multiple access situation, see Chang and
Wolf [8], to (M-1)ln2 bits per channel use if we assume no coordination
between users or one-to-one communication

Theorem 2.1.1 For the input probability distribution that puts a probability
1-(M-1)ln2/T on one symbol and a probability ln2/T on each of the
remaining (M-1) symbols, the total achievable rate R for T users becomes
(M-1)ln2 bits per transmission or (1-1/M)ln2 bits per dimension for T  .

Example In Figure 2.5a, for M = 2 the output is ternary: 0, 1, or (0,1). The

transitions are a result of the interference of the other users. Optimizing with
respect to p and letting T go to infinity, the achievable rate for this channel
becomes R(2,T)  ln2 bits per channel use, or a total of 0.5ln2 bits per
dimension. For each individual user, the 2-symbol input multiple access
channel is, from a rate point of view, equivalent to the binary input-binary
output Z-channel. Although the 2-symbol input multiple access channel has
a ternary output, the achievable sum rate is the same as if we make a hard
decision in case of an ambiguous reception of two symbols. The limiting

24
 Chapter 2-Random Access

input distribution for both examples puts all probability mass on one symbol.
For M = 2, P(0) = 1 - ln2/T, P(1) = ln2/T, respectively.

pT-1
0 0

1 1, (0,1)

Z-Channel

Figure 2.5a Binary Z access channel model

pT-1
p 0 0

(0,1)

1-p 1 1

3-ary output

Figure 2.5b Binary input ternary output access model

Remark The 3-ary output channel as given in Figure 2.5b looks like an
erasure channel, where the symbol pair (0,1) can be considered as the
erasure. One normally uses these channels with symmetric input probability
distribution to obtain channel capacity. However, in this case, the rate of the
channel would give a value of C = (1/2)T-1 for p = ½, which goes to zero for
large T. The reason for this discrepancy is the fact that the transition
probability is a function of the input probability. In information theory,
capacity is a convex cap function of the input probability distribution for
constant transition probabilities. In our situation, this is no longer the case
and we have to search for the global optimum. The problem is to express the
transition probabilities as a function of the input probabilities.

The M-ary Erasure-channel has as inputs Xi {0,1, ..., M-1}, and as outputs
Y {0, 1, ..., M-1, E}, where E occurs if two or more different symbols
occur at the output. We conjecture that the M-ary Erasure-channel as given

25
 Chapter 2-Random Access

in Figure 2.6 has a capacity of ln2 bits per transmission for T  . This
result was checked for M = 2, 3 and 4.

0 0

1 1
 

M-1 M-1

Figure 2.6 M-ary Erasure-channel

Example An example of a practical M-ary OR channel is that generated by

Pulse Position Modulation (PPM). Pulse position modulation divides the
transmission time into time slots of length τ and further sub-divides every
time slot into M sub-slots of time τ/M. An M-ary symbol is modulated as a
pulse in one of the M available sub-slots.

1 4 2 position in slot

/M 

Figure 2.7 Pulse Position Modulation for one user, M = 4

If we consider the M positions in a slot as M independent parallel channels,

the uncoordinated sum rate for T users is Runc(M,T) = ln2 bits per position.
For M positions we thus have a sum rate of M ln2 bits per slot. Note that the
number of possible different combinations in a slot is equal to 2M.

If we consider a time slot as a channel where we put a pulse in a particular

position, we have a total of M different inputs and the channel is equivalent
to the M-ary OR channel. The maximum number of different combinations
in a slot is now equal to 2M-1, since users have to give a pulse as input.
However, instead of using one out of 2M combinations of M symbols, the

26
 Chapter 2-Random Access

same rate can be achieved by using only a single symbol from M, which is
the advantage of the PPM modulation. The total achievable rate R for T
users becomes (M-1)ln2 bits per transmission, or (1-1/M)ln2 bits per
dimension for T  . A vanishing loss compared with the unrestricted case
for large values of M. To obtain this rate, we generate a pulse with
୑ିଵ
probability (1 – ln2) for one specific position in a time slot, whereas all
୘
other positions have a probability ln2/T of having a pulse.

For the transmission scheme of Figure 2.7, we may consider transmitting 1

pulse of duration /M in  seconds using a theoretical “bandwidth” of 2B =
M/ Hz. For M-ary Frequency Shift Keying (MFSK) we transmit one
frequency out of M possible frequencies with a duration of  seconds, using
a theoretical bandwidth of 2B = M/ Hz. Hence, from a “bandwidth” point of
view PPM and MFSK are equivalent, and the rate results apply to both
systems. Instead of using time, we can thus also use the frequency domain.

2.2 Signaling with Random Signatures

Access schemes for PPM using M-ary signatures can be found in [10,11]. In
Figure 2.8 we give the M-ary scheme as described by Cohen, Heller and
Viterbi [10], where an active user is allowed to transmit an M-ary signature
in L slots. Users are assumed to be frame aligned, i.e. every user knows the
starting time for the L slots. In general, the receivers look for a particular
signature connected with a message that belongs to their respective
transmitter. A decoder makes an error if it detects a valid signature that is
the result of the “OR“ of the other users. Codes are to be designed to
minimize the decoding error probability for a given number of active users.

L=3

1 4 2 position in slot

/M 

Figure 2.8 Access model with signature (1,4,2)

27
 Chapter 2-Random Access

2.2.1 Signatures with a symmetric input distribution

We first describe a transmission scheme using random signatures. We

assume that T users are connected with their T respective receivers. The
starting and end point of transmission is known to all transmitters and
receivers, and they are time aligned. The parameters of the transmission
scheme are:

- every user has its own set of M-ary signatures of length L. The
cardinality of the set is M;
- every symbol in a signature is chosen with probability 1/M and
corresponds with a pulse in the respective time sub-slot position.

The decoder for a particular user has to detect a valid signature that belongs
to the user. Suppose that a user transmits a particular signature. The
decoding error probability Pe for this particular user is upper bounded by the
probability that the T-1 other users together generate one of the M-1 other
valid signatures in L slots, i.e.

1 T 1 L
Pe  (M  1)  (1  (1  ) ) . (2.6)
M

For M = 2(1 - ε)L and T = M·ln2, the error probability

Pe  2 - ε L, ε  0,

and the normalized rate

log 2 M
=T  (1  ε)ln2 bit/transmission.
LM

In order to decrease the symbol error probability, we have to increase L and

thus, also M.

One sees from (2.6), that the average error probability for T users
simultaneously, TPe, does not approach zero for increasing L. If we change
the variables such that MT = 2(1 - ε)L and T = M·ln2, then the average error
probability for T users is given by

28
 Chapter 2-Random Access

TPe  2 - ε L, ε  0, (2.7)

but the normalized rate

log 2 M 1  ε
=T  ln2 . bit/transmission.
LM 2

This is roughly 50% of the maximum achievable rate.

2.2.2 Signatures with an asymmetric input distribution

We can improve the rate by using a different probability assignment and

signature scheme. For this, the signatures for a particular user consist of (M-
1) signatures with symbols not equal to 1. We reserve the all-1 signature for
message m = 1. In Figure 2.9a, we transmit the position 1 in three subsequent
time intervals. In Figure 2.9b we transmit a “spread“ sequence of three
positions.

L=3

M=4 time

Figure 2.9a Cohen, Heller and Viterbi model, the message is m = 1

L=3

M=4 time

Figure 2.9b Signature (2,4,3)

For a message probability

29
 Chapter 2-Random Access

୑ିଵ
Pr(m = 1) = 1 - ln2 = 1 – p,
୘

୪୬ଶ
Pr(m = k) = , 2  k  M,
୘

the amount of information that is transmitted per user is R = h(p) + plog2(M-

1), where h(p) is the binary entropy function. The decoder estimates m’ = k
if the valid signature for message k is detected in L slots. The decoder is
assumed to always decode message 1, unless another valid signature is
detected. The decoding error probability Pe for a particular user is then upper
bounded by the probability that the other users together generate one of the
other valid signatures in L slots, i.e.

1 pT 1 L
Pe  (M  1)  (1  (1  ) ) . (2.8)
M 1

We can approximate (2.8) as

Pe  M  (1  e pT/M ) L . (2.9)

୑ିଵ
For p = ln2, M = 2εL and T = 2(1-2ε)Lln2, the error probability (2.8)
୘
vanishes as

Pe  2 - ( 1 - ε )L, ε  0. (2.10)

From (2.10) it also follows that the average error probability for T users
simultaneously goes to zero for increasing L, since

TPe 2-ε L ln2  0, ε  0. (2.11)

At the same time the normalized rate

h(p)  plog 2 (M  1)
 = T×  (1 - 2) ln2 bit/transmission (2.12)
LM

The signature scheme with an asymmetric distribution clearly performs

better than the signature scheme with a symmetric distribution. As before,
one position is selected as a common position for all users.

30
 Chapter 2-Random Access

Remark The asymmetric input probability distribution scheme has M = 2εL.

We can keep the value of M constant by choosing ε = O(1/L) leading to a
vanishing probability of error and also a constant M.

2.3 Signaling with Constructed Signatures

2.3.1 The Titlebaum construction

The first signature construction we discuss is the Titlebaum [13]

construction. Suppose that we extend the first two rows of an RS encoding
matrix as

1 1 1 1 … 1
G2,M = . (2.13)
2 M-2
0 1   … 

It is easy to see that the minimum distance of the code is M-1, where M is
the length of the extended code. The number of different non-zero code
words formed with the second row only is M-1. These code words are used
as signatures for M-1 different users. We assume that the users are fully
synchronized. In addition, every user may send one of M messages by
adding a multiple of the first row to his signature, and create a modulated
signature. At the receiver side, we assume error free detection of the
transmitted symbols. For the encoding matrix as given in (2.13), we have the
following properties:

- the Hamming distance between the modulated signatures for a particular

user is M;

- the Hamming distance between signatures for two different users is at

least M-1.

Since the dmin  M-1, the maximum number of agreements between two
signatures from different users is equal to 1. Consequently, M-2 or less
active users can never produce a valid signature for another particular user,
since at least one symbol is left for the unique identification.

31
 Chapter 2-Random Access

Example For M = 5, we can form the 4 signatures: C1 = (0,1,2,3,4), C2 =

(0,2,4,1,3), C3 = (0,3,1,4,2) and C4 = (0,4,3,2,1). These 4 signatures have a
minimum distance of 4 and are used as the signatures for 4 different users.
From these signatures we can easily form the set of 5 possible transmitted
modulated signatures for a particular user. For user 1, with signature
(0,1,2,3,4) these are

C = {( 0,1,2,3,4 ), ( 1,2,3,4,0 ) , ( 2,3,4,0,1 ), ( 3,4,0,1,2 ), ( 4,0,1,2,3 )}.

Remark If we shorten the signatures from length M to length L, the

minimum distance within a set of signatures is reduced from M to L and the
minimum distance between different sets of signatures is reduced from M-1
to L-1. Shortening can be used to influence the efficiency of the modulation
scheme. Therefore, we can give the normalized efficiency or rate of a
transmission scheme using the Titlebaum construction for T active users as

log 2 M
T .
LM

The receiver consists of a bank of M energy detectors matched to the M

chips of the M-ary signaling scheme. The M energy detectors provide ML
outputs per received word. The decoder for a particular user compares the
demodulator output with its M possible transmitted signatures. It outputs the
signature for which the maximum number of agreements with the symbols at
the demodulator output occurs (minimum distance decoding).

2.3.2 Decoding error probability for the Titlebaum construction

We consider the situation when only user interference occurs. Furthermore,

we assume that the demodulator for a particular user always outputs the
transmitted symbols. These assumptions make the analysis easier. Due to the
structure of the demodulation process, decoding errors may occur if and only
if the other users together cause the occurrence of a modulated signature for
a particular user.

Since the minimum distance between signatures is L-1, a signature for a

particular user interferes in exactly one position with a signature from
another user. This enables us to calculate the error probability. For T active
users, an error may occur if at least L out of T-1 users together generate an

32
 Chapter 2-Random Access

additional valid signature for the remaining active user. Assuming random
selection of the messages, this probability is given by

 T  1
Pe  ( M  1)[( M1 ) L     L!]. (2.14)
 L 

This follows from the fact that, for a particular set of L users, L!
permutations can give a particular signature with probability (1/M)L.
Furthermore, we can choose L out of T active users that will produce the
particular signature.

In Figure 2.10, we plot (2.14) for several values of M, T and L.

1.0

Pe
10--1
M = 256, L = 10

10--2

M=32, L=6

10--3
.05 .06 .07 .08 .09 0.1
log 2 M
T
LM

Figure 2.10 Pe as a function of the efficiency 

Note that for large T and small L, (2.14) can be approximated as

Pe  M( T ) L . (2.15)
M

The asymptotic behavior of (2.15) is the same as that of (2.6) where we use
random signatures. The above transmission scheme assumes that the number
of active users T < M. Extension to T > M is straightforward.

33
 Chapter 2-Random Access

2.3.3 Superimposed Codes

Kautz-Singleton (KS) [11] introduced the class of binary superimposed

codes (SIC) to construct signatures. A binary superimposed code SIC(U, n,
2, T) consists of U binary code words of length n, with the property that the
Boolean sum (OR) of any T-subset does not cover any word not in the T-
subset.

We extend the definition of binary SICs to the situation where code words
have q-ary symbols and the channel output is a symbol which identifies
which subset of symbols occurred as input to the channel (no how many of
each). We first have to give some additional definitions.

Definition The union of a set {a,b, ···,c} denoted as (a,b, ···,c) is defined as
the set of different symbols of the argument (a,b, ···,c).

Example (1,2,3,3,2) = {1,2,3}.

Example  (0,1,1,0,0) = {0,1}.

Let V  { 0, 1, ···, (q-1) }n, V = U, be a code book with U q-ary code
words.

Definition The union of T code words in V, n (rn, sn, ···, tn) is defined as
the component wise  of the symbols.

Example 4(1223,1321,1111) = ({1},{1,2,3},{1,2},{1,3}).

Definition The union n of T code words n(rn, sn, ···, tn) covers a code
word vn if

n(rn, sn, ···, tn) = n(rn, sn, ···, tn, vn).

Example The word ({1},{1,2,0},{1,0}) covers the code word vn = (1,0,0).

Definition A q-ary superimposed code with parameters U, n, q, T contains U

q-ary code words of length n with the property that the n of any set S
containing T or less code words does not cover any code word not in S.

34
 Chapter 2-Random Access

The following proposition then follows.

Proposition

T
 U
q
(2  1)  n
 i . (2.16)
i 1

Another lower bound on n can be obtained as follows. We first consider (for

simplicity) the situation where n = T·s.

n = T·s
We partition the code words into T parts of length s. Every code word must
have a part different from the corresponding part of all other code words.
This part contains at least one symbol, called special element, that can be
used to distinguish a code word from the union of any set S of T or less code
words. If the number of special elements in a particular part is exactly qs, we
have U = qs. We must therefore assume that every part contains at most qs-1
special elements. The maximum number of different parts is an upper bound
for the number of code words in the code, and thus U  T(qs-1). Since n =
Ts we obtain

T U
n log 2 , (2.17)
log 2 q T

which improves on (2.16).

We can also give a lower bound for n using a probabilistic approach.

Consider a U ˟ n matrix containing q-ary elements that are uniformly and
independently distributed. An error occurs if a selection of T or less rows
covers a row not in the selection. For a q-ary SIC the probability that this
happens can be bounded by

n
 U   q  1  
T

Pe(q-ary SIC )  (U  T)  1     . (2.18)

 T    q  

For T = q ln2, n(1+) > Tlog2U, the probability Pe(q-ary SIC ) → 0 for
increasing n.

35
 Chapter 2-Random Access

T>n
For T > n, every code word must have a special element in at least one of its
positions. Again, if one of the positions for all code words contains exactly q
special elements, then U = q. Therefore, every position must contain no more
than q-1 special elements. Hence, we obtain as a lower bound for n,

U  n(q-1).

Example The following example gives a q-ary SIC(U = 15, n = 5, q = 4, T),

where n  T < n(q-1) and U = n(q-1). The SIC(15, 5, 4, T) contains the
following code words

10000 01000 00100 00010 00001

SIC(15, 5, 4, T) = 20000 02000 00200 00020 00002 .
30000 03000 00300 00030 00003

The example can easily be generalized to other values of n and q.

We now consider the construction of q-ary SICs using RS codes due to

Kautz-Singleton [11]. The parameters for these codes are:

- RS “outer“ codes over GF(q) with length n = q-1, dimension k,

minimum distance dmin = n-k+1;
- a mapping of every RS code symbol to a binary word of length q and
weight 1.

Note that the number of disagreements between two code words is larger or
equal to dmin and thus, the number of agreements is less than or equal to n -
dmin = k – 1. As a consequence, the Boolean sum of T code words agrees
with an arbitrary code word in at most T(k-1) positions. If T(k-1) < n, we
have the following theorem.

Theorem 2.3.1 There exists a

n
SIC (U  q k , n KS  q(q  1), q, T  ).
k 1

We compare the KS codes in rate with TDMA for T active users. We give
every user a set of qi code words out of qk. The number of potential users is
thus qk-i. If a SIC exists, the normalized rate is

36
 Chapter 2-Random Access

T log 2 q i
KS  .
q(q  1)

For TDMA every user gets a slot where a symbol can be sent. The rate is

T
ηTDMA  .
q k i

We can see that, for k > i + 1, the KS codes are superior in rate.

Example Let k = 4, i = 2, q = 25. For these parameters:

 T = 7 < n/(k-1);
 The number of possible users is q4/q2 = 210 ;
 The rate ηKS ≈ 0,08; ηTDMA ≈ 0.007..

Theorem 2.3.2 The extended RS code with parameters (n = qs, k = qs-1, dmin
= n –k + 1) code over GF(qs), where q is any prime and T  q, defines a
superimposed code SIC(qsk, qs, qs, T).

Proof It is easy to check that for T  q, the condition of Theorem 2.3.1 is

fulfilled.

Remark The condition on T in Theorem 2.3.1 is sufficient but not necessary

for the existence of a q-ary SIC. This follows from the next example.

Example Let q = 3, T = 2 and n = 4. The following code has minimum

distance 2. The corresponding q-ary SIC

0 0 0 0 0 1 1 0 0 2 2 1 11 2 2

q-SIC(12, 4, 3, 2) = 1 2 0 1 1 0 1 0 2 2 1 1 2 0 2 1

2 1 0 1 2 2 2 0 0 0 1 2 2 2 0 2

does not satisfy the condition on T in Theorem 2.3.1.

37
 Chapter 2-Random Access

The following theorems construct longer codes from shorter ones.

Theorem 2.3.3 If there exists a SIC(U0, n0, q0, T) and a SIC(U1, n1, q1, T),
where q1  U0 , then there also exists a SIC(U1, n0 n1, q0, T).

Proof Assign to each symbol {0, 1, ···, q1-1} a different code word from
SIC(U0, n0, q0, T). Replace the symbols in SIC(U1, n1, q1, T) by these code
words. Since we replaced all q1-ary elements by different code words from
SIC(U0, n0, q0, T) we thus obtain a SIC(U1, n0 n1, q0, T), see also Figure 2.11.

SIC(U0, n0, q0, T) SIC(U1, n1, q1, T),

U0
U1

n0

n1

SIC(U1, n0 n1, q0, T).

U1

n0 n0 ••• n0

n0 ˟ n1

Figure 2.11 Construction of a SIC out of two others

Corolary If there exists a SIC(U0, n0, 2, T) and a SIC(U1, n1, q1, T), where
q1  U0, then there also exists a SIC(U1, n0 n1, 2, T).

The codes constructed in Theorem 2.3.3 can be seen as a generalization of

the Kautz-Singleton codes [11].

38
 Chapter 2-Random Access

Example
1. Suppose that we have the following starting SIC(4, 3, 2, T = 2 ) with the
4 code words

(1 0 0, 0 1 0, 0 0 1, 1 1 1 )  ( 0, 1, a, b ).

2. The second code to be used is an RS code over GF(22) with parameters

(n, k, dmin) = (3, 2, 2). For this code, T < n/(k-1). Hence, we can
construct a SIC(16, 3, 22, 2). We replace every element with a code
word from the first code and obtain a SIC(16, 9, 2, 2) with 16 code
words, i.e.

000, 01a, 0ab, 0b1, 1a0, ab0, b10, a01

SIC(16, 9, 2, 2) = .
b0a, 10b, 1ba, a1b, ba1, 111, aaa, bbb

3. As a third code we construct an RS code over GF(24) with parameters (n

= 15, k = 8, dmin = 8), where T < n/(k-1). From this code we obtain a
SIC(232, 15, 24, T = 2). For the combination with the second code we
obtain a SIC(232, 9×15 = 135, 2, 2).

Example Let q = 4 and T = 3.

1. The first code we use is an RS code over GF(22) with parameters (n = 4,
k = 2, dmin = 3). Since T < n/(k-1), we obtain a SIC(24, 4, 22, 3).
2. The second code we choose is a shortened RS code over GF(24) with
parameters (n = 13, k = 5, dmin = 9). Since T < 13/4, we obtain a SIC(220,
13, 16, 3).
3. Combining both codes, we obtain a SIC(U = 220, 4×13 = 42, q=4, T = 3).

These examples show that we can construct a series of codes, see also [12].

39
 Chapter 2-Random Access

2.4 Signatures for the XOR Channel

We consider the situation where T active users from a population of size U
transmit signatures over the XOR multiple access channel. The symbols in a
signature are elements from GF(q = 2m). We furthermore assume that the
users are block synchronized.

U – k columns

user i siU-k

user j sjU-k

··· ··· U rows

user h shU-k

zU-k = siU-k + sjU-k + shU-k

Figure 2.12 RS syndrome former used for random access

Every user has a unique q-ary signature siU-k, 1 ≤ i ≤ U, that corresponds to a

particular row of the parity check matrix HT of a (U,k) RS code over GF(2m)
with (U - k) columns and U rows. We use the property that for a minimum
distance dmin = (U – k + 1) any (U – k) rows of HT are linearly independent.

Suppose that we have a set S of T = pU active users, each transmitting their

signature multiplied with the information ti, ti ϵ GF(2m), ti  0. At the
receiver we have

zUk   t isi U  k ,
iS

where the addition is in GF(2m). For T ≤ (U – k)/2 active users, the receiver
uses the RS decoding algorithm, and decodes the information (noise)
transmitted by the active users. Hence, the maximum sum rate becomes

40
 Chapter 2-Random Access

(U  k)log2 (q  1) 1
η  bit/transmission.
2(U  k)m 2

In case the decoder knows the active users, the transmission efficiency η = 1
bit/transmission, since every user can transmit log2q bits/transmission. This
is the maximum possible over this binary output channel. We visualize the
access scheme in Figure 2.12.

We calculated the achievable rate for this channel in the same way as we did
for the OR channel. For a particular user, we have a binary symmetric
channel with crossover probability

Pe = ½ ( 1 – (1 – 2p)T-1),

where p is the probability of an input symbol equal to 0. Note that for p = ½,

Pe =1/2 and the normal channel capacity is zero. We have to assume an
asymmetric distribution for p to improve the result. The following
achievable rate was obtained.

R XOR  lim T max I(X i ; Y)

T  P(X i )

 1  e  2c 
 ce  2c log 2    0.3138 bits/trans mission.
 1  e  2c 
 

for the input probability distribution

p = P(0) = c/T; P(1) = 1 - c/T; c = 0.1999.

Observe that a positive rate remains for T  . For more information about
the applications and background of the XOR channel, we refer to
[80,81,82,83].

2.5 Concluding Remarks

In this chapter, for which we summarize the concept in Figure 2.13, we start
with calculating the maximum achievable transmission efficiency

41
 Chapter 2-Random Access

(throughput) in a random access situation. The considered problem is

different from the classical information theory problem, in the sense that
users do not know each other and thus cannot cooperate. Furthermore, in a
multiple user situation, the interference depends on the channel input
probability distributions and thus classical maximization methods using
concavity arguments to obtain channel capacity do not work.

capacity/throughput
OR channel

capacity/throughput
M-ary OR channel

random signatures

signature construction

Figure 2.13 Concept of the chapter

For the binary OR channel and for the M-ary OR channel, the throughput
approaches ln 2 bits per channel use. This throughput has to be shared by the
number of active users. The loss in efficiency, compared with fully
coordinated access, is about 30%. Remark, that the M-ary OR channel is
equivalent to the Pulse Position Modulation scheme. By using only one
pulse in M time slot, we obtain the same throughput as if we are using M
parallel binary OR channels. These topics were considered by Jeroen
Keuning2, a student form the University of Eindhoven, the Netherlands, and
Peter Gober3 a PhD student from the University of Duisburg-Essen.

An interesting approach was taken by Young-Gil Kim in [85], where the OR

channel model was used to improve a collision arbitration algorithm in RFID
(Radio Frequency Identification) applications.

42
 Chapteer 2-Random A
Access

For the M M-ary OR channel,

c we develop
d andd analyze a trransmission scheme
using siggnatures. Forr random siggnatures, wee show that ana asymmetrric input
distributtion improvees the throuughput. This implies thaat a particullar user
almost aalways transmmits a pulse in
i a “central”” position. So
ometimes on ne of the
remaininng time posiitions is useed. We alsoo indicate th he problem of o error
probability for a set of
o active useers simultaneeously. For th
he random siignaling
scheme, a throughpuut of (½ln2) bits per chaannel time un nit can be acchieved.
We concclude with tw wo constructtions for siggnature sequeences. One is called
the “Titllebaum” connstruction, based
b on an extended RS R encoding matrix.
Togetherr with Samw wel Martirosssian4, who w was a guestt in the Instiitute for
Experimmental Mathhematics, we w further developed the conccept of
“superimmposed codess.” From thiis concept a sseries of coddes can be derived.

We concclude the chhapter with the

t constructtion of signatures for thhe XOR
channel. This researrch was initiated by T. EEricson and V.
V Levenshteein [80].
V. Levennsthein was a visitor of the Institutee for Experimmental Math
hematics
and we ppublished a paper
p on peakk-shift correccting codes [84].
[

In inform
mation theoryy, network users
u are assuumed to be able
a to coopeerate. In
certain network situuations this is indeed possible. The T main prroblems
considerred are the achievable raate regions annd their conv
verses. Exam
mples in
our reseaarch can be found
f in [90,,91].

k Wolf1 at th
The repoorted researcch was inspirred by lecturres from Jack he 1981
Nato Addvanced Instiitute in Norw
wich, Englandd.
1
J.K. Skwwirzynski, Neww Concepts in Multi-User
M Commmunication, Norwich,
N UK, August
A 4-
16, 1980
2
A.J. Haan Vinck and Jeroen
J Keuningg, „On the Cappacity of the Asynchronous
A T-user
T M-
frequency Noiseless Multiple
M Access Channel w without Intensitty Information n,“ IEEE
Transactioons on Informattion theory, Nov
v. 1996
3
P. Goberr. Aspekte des unkoordinierten
u n Mehrfachzugrriffs, Verlag disssertation.de, 20
000.
4
A. J. Haan Vinck, Samvvel Martirossiann and Peter Goober, “On q-aryy Superimposed d Codes,”
1999 IEEE E ITW, Kruger National Park, South Africa, JJune 20 – 25, 19999

Professoor Jack Wolf (1935-2011)

43
Chapter 2-Random Access

44
 Chapter 3-Cooperative Coding

Chapter 3

Cooperative Coding
3.1 Introduction
One of the dominant parameters in power line communications is that of
attenuation. Attenuation can vary between 10-100 dB per kilometer. In
power line communications, attenuation prevents signals to propagate over
distances that are longer than say 500 meters. Investigations to overcome the
high attenuation are therefore highly important, see [14,15,16].

f f f

•••
T

N(0,2) N(0,2) N(0,2)

R1 R2 R

Figure. 3.1 L links connecting T and R

45
 Chapter 3-Cooperative Coding

We discuss the efficiency of communication over links that are in tandem

[17]. This situation occurs when users in a power line communication
network are connected to the same line and the power lines can be seen as
“bus systems”, where all connected users can listen to the signals present on
the bus. An injected signal can be observed by all connected users.

Figure 3.1 gives a model for L links connecting T and R using the power
line. We assume for simplicity that all links have the same length, same
attenuation as well as the same noise variance. The attenuation factor f is a
function of the length of a link, i.e. f = f(d), where d = D/L and D is the
distance between transmitter T and user R.

To illustrate the problem, consider an AWGN channel with signal

attenuation factor f and PSK modulation with Eb the energy per information
bit and N0/2 the noise variance. The detection error probability at the
receiver

 Eb 2  E
P1  Q f   exp( b f 2 ) , (3.1)
 N  2N0
 0 

where the Q-function is the tail probability of the standard normal

distribution. If we have L links in tandem, then the attenuation is f and the
detection error probability after L links becomes

 E b 2L  2( L 1)
P2  Q f   exp( E b f 2L )  P1f . (3.2)
 N  2 N
 0  0

We can see a dramatic degradation in performance for f < 1, even with only
two links. To overcome this degradation, we can use the fact that every user
can observe the signal and thus can possibly act as a repeater or relay. We
investigate several possibilities in Section 3.2.

3.2 Communication Strategies

In the following, we discuss possible communication strategies to efficiently
use - and operate repeaters under the condition that not more than one user
can use the bus simultaneously. The first two strategies fall into the category
of conventional multi-hop, while the third makes use of incremental
redundancy.

46
 Chapter 3-Cooperative Coding

Detect-and-forward
In the connection between the transmitter T and receiver R, the intermediate
receivers can act as detect-and-forward repeaters. After taking a hard
decision on the received signal, repeater i sends the re-modulated signal to
receiver i + 1. Therefore, if at the first repeater the detection error probability
is Pe, then the error probability at the Lth receiver is approximately LPe.
While this linear increase of error probability with distance is far better than
the exponential growth experienced without repeaters, the use of repetitions
reduces the overall transmission efficiency by a factor of L. Since Pe is
determined by the attenuation factor f(d), and d = D/L, the number and
location of repeaters can be optimized such that L is minimized subject to an
upper bound on the error rate LPe at the destination.

Decode-and-forward
As in point-to-point connections, we can also use coding to improve the
communication efficiency. Given the model described above, we have the
capacity C(d) = 1 – h2(Pe) for a link of distance d, where h2(·) is the binary
entropy function. Assuming that C(d) is a measure for the achievable rate
with coding, which is monotonically decreasing with increasing d, and
considering that L hops reduce the overall achievable rate to

Cdf = C(d)/L, (3.3)

we again have an optimization problem for the number and thus location of
repeaters.

Cooperative coding
Since we have a bus structure, we can use the fact that all users connected to
the bus are able to hear the communication and cooperate to improve the
communication efficiency, see also [19]. The achievable rate for the
communication between transmitter T and the first receiver is k/n1 = C(d),
i.e., after n1 transmissions the first receiver can decode the transmitted k bits
of information. Simultaneously, the amount of information received at the
second receiver is n1C(2d). To assist the second receiver in decoding the
transmitted message, repeater 1 sends n2 encoded symbols, such that k =
n1C(2d)+n2C(d). For the third transmission, repeater 2 transmits n3 encoded
symbols such that k = n1C(3d)+n2C(2d)+n3C(d) and the third receiver can
decode (in theory) the message. For the destination, we have

47
 Chapter 3-Cooperative Coding

L
k  n i C((L  i  1)d). (3.4)
i 1

In summary, given k and the link capacities C(id), the lengths ni, i = 1, 2, ···,
L, are obtained from

 C( d ) 0 0   n1   k 
 C(2d ) C( d ) 0 n   k 
  2      (3.5)
          
    
C( Ld) C((L  1)d )    C(d ) n L   k 

The overall achievable rate is

k
C coop  . (3.6)
L
 ni
i 1

If we do not use the cooperative coding, we have from (3.3),

Cdf = k/(Ln1).

Example Let us consider L = 2. With decode-and-forward the rate is Cdf =

½C(d) = k/(2n1). With cooperative coding we obtain

k C(d ) 1
C coop    C(d ). (3.7)
n1  n 2 C ( 2d ) 2
2
C( d )

Clearly, for f = 1, C(2d) = C(d) and thus n2 = 0 and the repeater is not used.
However, for f < 1, C(2d) < C(d), and cooperative coding has an advantage.

Example If for example, Pe = 10−3 and f = −10 dB, then C(d) = 0.99
bit/transmission and C(2d) = 0.36 bit/transmission. Hence, cooperative
coding increases the rate from 0.36 bit/transmission using no repetition or
0.49 bit/transmission using decode-and-forward to C(d)/(2 − C(2d)/C(d)) =
0.60 bit/transmission.

48
 Chapter 3-Cooperative
3 e Coding

Figure 3.2 illustrates the bus sysstem for L = 2 as a relay channel. To o outline
the advaantages of coooperative cooding, considder a binary modulation scheme
with a trransmitter sidde SNR of 131 dB, and a T-R distancce of D = 1k km. The
attenuatiion for a linkk of length d = D/L is ggiven by f2 = 10-δd/10, whhere we
choose δ = 40, 60, 1000 dB/km.

relay
n1C(d) n2C(d)

T R
n1C(2d)

Figure 3.2 The

F T two link
k connection as a relay ch
hannel.

The erroor probabilityy for direct transmission

t n over i linkss follow from
m (3.2),
and the correspondinng link capaacity as C(i × d) = 1 – h2(Pi). We th hen can
computee the achievaable data rate for decodde-and-forwaard and coop perative
coding ffrom (3.3) annd (3.6), resspectively. F Figure 3.3 shhows the ach hievable
rate as fuunction of thhe number off repeaters N..

40

60

100

FFigure 3.3 Achievable

A rate
r for coopperative codiing and deco
ode-and-
fforward

49
 Chapter 3-Cooperative Coding

We observe that cooperative coding achieves a higher rate for a given

number of repeaters. In particular, conventional multi-hop reaches a
maximum rate at a certain repeater distance d = D/N, which decreases when
repeater distance decreases through the addition of repeaters. Cooperative
decoding, however, experiences a continuing increase in achievable rate for
increasing N and thus decreasing repeater distance d.

Example We use the (7,4) Hamming code to illustrate an implementation of

cooperative coding with a fixed-rate code. Suppose that we have a bus with
only L = 2 links. The length of the links is such that we expect a single error
in a received code word per link. For the transmission using two links in
tandem via detect-and-forward we expect two errors to occur, which cannot
be corrected using the Hamming code. Applying decode-and-forward would
allow error free transmission. But in both cases, the rate is only 4/14 bits per
transmission. For cooperative coding, we use the Hamming code generated
by the encoding matrix

1 0 0 1 1 0 0
0 1 0 1 0 1 0
G = . (3.8)
0 0 1 0 1 1 0
 
1 1 1 1 1 1 1

The reason for this choice of encoding matrix will be made clear in the
following.

1. The first receiver can decode the transmitted code word if it has no more
than a single error. The second receiver also receives a signal, but it
cannot reliably decode the transmitted code word since the number of
errors might be above the correcting capacity.
2. The first receiver (repeater), after decoding, encodes the first three
information bits with a minimum-distance 3 shortened Hamming code of
length 6, generated by the code words [100110], [010101], [001011]
(see (3.8)).
3. The second receiver can decode the received code word, since it contains
no more than a single error. After decoding, the second receiver can
subtract the code word connected to these three information bits from
the first reception and decodes the fourth information bit using the

50
 Chapter 3-Cooperative Coding

minimum distance 7 code generated by [1111111] which is able to

correct three errors. The overall efficiency is 4/(7 + 6) > 4/14.

The example shows the importance of the encoding matrix representation.

Following this strategy, the optimum distance profile (ODP) of linear block
codes is defined in [18]. Using the ODP, for particular codes, we are able to
specify the maximum minimum distance of a sub-code. The ODP depends
on the initial encoding matrix, as in the example.

The extension to RS codes can be explained using the k/n RS encoding

matrix

1 1 1 ... 1
1 α α2 ... αn−1
G= 1 α2 α4 ... α2(n−1) , (3.9)
...
1 αk−1 α 2(k−1) ... α (k−1)(n−1)

where the α is a primitive element from the GF(2m) and n ≤ 2m − 1. Suppose

again that L = 2 and that we start with the k × n encoding matrix G at T. The
repeater has an encoding matrix generating the (m2,k2) RS code consisting of
the top k2 rows of length m2 of G, where m2 < n. T encodes k symbols and
forwards the code word to the repeater. The repeater can decode the received
word from T, whereas R2 can listen but not decode since the reception is too
noisy. The repeater then encodes the first k2 symbols, and forwards the code
word that corresponds to the (m2,k2) RS code. R2 is assumed to be able to
decode these k2 symbols and subtracts the influence from the first received
code word. Since the minimum distance of the remaining code is increased,
R2 can decode the whole code word. The performance of this method
depends on the knowledge of the respective channel parameters and can be
optimized for particular values of the channel parameters.

k2

m2
k1

Figure 3.4 Dimensions for the RS encoding

51
 Chapter 3-Cooperative Coding

Suppose that the link error rate is upper bounded by t1 and the two link error
rate is upper bounded by t2, where ½ > t2 > t1. The dimensions of the RS
canonical encoding matrix that we are going to use are given in Figure 3.4.
The relations between the dimensions and error rate are as follows:

n – (k1 + k2) = 2t1 n, (3.10a)

n – k1 = 2t2 n, (3.10b)
m2 – k2 = 2t1 m2. (3.10c)

The encoded information to be transmitted to user R2 is given by k = (k1 +

k2) symbols. The communication uses three steps.

1. In the first step we encode k information symbols and transmit the n

code symbols to R1. Using (3.10a), R1 can decode. Receiver R2 receives
a noisy version of the code word with a maximum of t2 ˟ n errors, which
is assumed to be beyond the error correcting capability of the code with
the given parameters.
2. In the second step, receiver R1 forwards the code word, that corresponds
to k2 information symbols, of length m2. Using the RS encoding matrix
properties, this is again an RS code word.
3. From (3.10c), it follows that R2 is able to decode the k2 information
symbols. Since R2 can subtract the influence of the decoded information
from the code word in the first step, R2 is able to decode the remaining
k1 information symbols using property (3.10b).

We can evaluate the transmission efficiency for simple repeating and RS

encoded retransmission as

k
R REP (T  R 2 )  ,
2n
(3.11)
k
R RS (T  R 2 )  .
2t  2t1
nn 2
1  2t1
Since ½ > t2 > t1, we can see the improvement that depends on the error rate
t2 and t1. For t2 close to t1 the efficiency is close to the situation without
repeater. For t2 close to ½, the un-coded repeating efficiency appears.

Remark For error rates larger than t2 or t1, we accept decoding errors.

52
 Chapter 3-Cooperative Coding

3.3. Multi User Tandem Communication

In this section, we first look at the communication from one transmitter to
two receivers (see Figure 3.5a). We give the efficiency for time-sharing as a
reference value for further comparisons and we use the water-filling
argument [22] to find the optimum frequency-sharing power allocation. We
use principles from information theory to translate the tandem channel
into a degraded broadcast channel and calculate its performance [20]. We
show that this translation improves the performance compared with the time-
and frequency-sharing results.

f f

•••
T

N(0,2) N(0,2)

R1 R2

Figure. 3.5a One transmitter to two receivers.

In Section 3.2 we describe the use of a repeater and its consequences for the
transmission efficiency. It is interesting to compare the repeater strategy with
the broadcast results and determine the condition for which it is not
beneficial to use a repeater. Since practical coding to achieve the capacity
region for broadcast channels is not a solved problem yet, we conclude that
for the tandem channel without repeater, time-sharing is the best option.

Finally, we introduce a multiple access model for the communication

between two transmitters and one receiver using the tandem channel model,
see Figure 3.5b. Information theoretical principles are used to show
equivalence between the broadcast and the multiple access models.

53
 Chapter 3-Cooperative Coding

3.3.1 Point-to-point communication

In this section we consider only two links and study several communication
topologies. For simplicity, we assume that the attenuation factor f, f < 1, is
the same for both links. The noise is considered to be additive white
Gaussian noise (AWGN) with power spectral density 2 W/Hz and average
value 0. The bandwidth and transmit power are denoted by B and P,
respectively.

The Gaussian channel capacity C1 for a single link and C2 for 2 links in
tandem, are given by

 Pf 2 
C1  Bln 1  nat/s,
 2σ 2 B 
 
(3.12)
 Pf 4 
C 2  Bln 1  nat/s,
 2σ 2 B 
 

respectively. All further derived transmission efficiencies (capacities) are

expressed in a similar way.

T2 T1
f f

•••

N(0,2)

R
Figure 3.5b Two transmitters to one receiver.

54
 Chapter 3-Cooperative Coding

3.3.2. Communication strategies

A. Time-sharing.
If we use time-sharing (TS) with time-sharing parameter  from transmitter
to both the receivers, then the achievable communication rate or
transmission efficiency is given by

 Pf 2 
R TS (T  R1 )  αBln 1  nat/s,
 2σ 2 B 
 

 Pf 4 
R TS (T  R 2 )  αBln 1  nat/s, (3.13)
 2σ 2 B 
 

0  α  1; α  1 - α.

B. Frequency-sharing.
For frequency-sharing, we use the water-filling (see Appendix 8.8) concept
from information theory to obtain the power allocation that maximizes the
efficiencies. We divide the frequency band into the part α and β, α + β ≤ 1.

 P f 2 
R FS(T  R1)   Bln 1  1 nat/s,
 2σ2B 
 
(3.14)
 P f 4 
R FS(T  R 2 )  Bln 1  2 nat/s,
 2σ2 2B 
 

where P = P1 + P2. To obtain the efficiency region for the frequency-sharing

using the water-filling argument, we have to maximize (3.14) for all
different values of α and β, α + β ≤ 1 under the sum power constraint. This
problem was solved in [99]. The key point in the optimization is the
assumption that α + β can be smaller than 1. The conclusion is that the
envelope of all solutions dominates the time sharing performance. The
general behavior is given by the dashed line in Figure 3.8. Analyzing the
results from A and B, we conclude that frequency-sharing is always better
than time-sharing. Note that both efficiency regions have the same boundary
points. For α + β = 1, we can use the approach from Appendix A.8.3.

55
 Chapter 3-Cooperative Coding

furthermore, for P1 = αP and P2 = (1- α)P, we obtain the “naïve” time-

sharing result as given in 3.13.

C. Receiver R1 acts as relay.

If receiver R1 can act as an intelligent relay, a simple decoding and re-
encoding strategy would give a transmission rate region

 Pf 2 
R REP (T  R1)  αBln 1 
 2σ 2B 
 
 αC1 nat/s,
(3.15)
α  Pf 2 
R REP (T  R 2 )  Bln 1 
2  2σ 2 B 
 
α
 C1 nat/s,
2

where  acts as a time-sharing parameter. Comparing the limiting points,

i.e. for  = 0 or for  = 1, we can conclude that repeating is worse than time-
sharing for

P 1  2f 2
 .
σ2B f6

We see that in a situation where power is large and bandwidth is small, time-
sharing is better than repeating. For small f, repeating is preferable.

Example: For σ2 = 10-8 W/Hz and B = 105 Hz, P = 25 W, an attenuation

factor f > 0.2 satisfies the condition.

We show in [17] and (3.7), that the performance for the repetition strategy
can be improved to

C1
R *REP (T  R 2 )  (1  α) nat/s. (3.16)
C2
2
C1

56
 Chapter 3-Cooperative Coding

In general, this improved repetition strategy needs a complex encoding and

decoding strategy. In a practical situation, we can use RS codes.

D. The Gaussian Broadcast channel.

The two link model can be transformed into a broadcast model, i.e.

y1 = x f + n1,
y2 = x f2 + n2,

where n1 and n2 represent the Gaussian noise with variance 2 and η2,
respectively. The input average power constraint is given by E(X2)  P. The
capacity region for the Gaussian broadcast channel has been determined [22]

f n1 N(0, 2 )

y1 R1
x

x
y2 R2

f2 n2 N(0, σ2)

Figure 3.6 The 2-link broadcast model.

and is given by

 γPf 2 
R BC (T  R1)  Bln 1   nat/s,
 2σ 2B 
 
(3.17)
 γPf 4 
R BC (T  R 2 )  Bln 1   nat/s.
 2σ 2 B  γPf 4 
 

Note that for (,) = (1,1) or (0,0) we have the same result for the broadcast
efficiency as for time- or frequency-sharing. However, simple calculations

57
 Chapter 3-Cooperative Coding

show that the capacity region for the broadcast channel is strictly larger than
the frequency- and time-sharing result for all other values of . The rate
regions approach each other for B  . This conclusion can also be found
in [21].

Example For σ2 = 10-8 W/Hz, B = 105 Hz, P = 25 W, f = 0.3, α = 1/2 and γ

= 0.02 we obtain RTS(T→R1) = RBC(T→R1) but RBC(T→R2) ≈ 2 RTS(T→R2).

Hence, we obtain a clear improvement of the broadcast efficiency over time-

sharing. The challenging problem is to design practical coding strategies
that achieve this gain in efficiency. This is still an open problem.

In [20] a general tandem channel model with more than 2 receivers is given.
In this case the broadcasting capacity is given by

 α Pf 2 
R BC (T  R1)  Bln 1  1  nat/s,
 2σ 2 B 
 

 α 2 Pf 4 
R BC (T  R 2 )  Bln 1   nat/s.
(3.18)
 2σ 2 B  α Pf 4 
 1 

 α3Pf 6 
R BC (T  R 3 )  Bln 1   nat/s.
 2σ 2B  α Pf 6  α Pf 6 
 1 2 
 ,

where α1 + α2 + ··· + αN = 1. Calculation of the capacity then becomes an

optimization problem.

E. The multiple access channel.

The two transmitter one receiver concept of Figure 3.5b, can be transformed
into a multiple access channel as shown in Figure 3.7. The output y at the
receiver can be written as

y = f x1 + f2 x2 + n,

58
 Chapter 3-Cooperative Coding

where n is the additive white Gaussian noise with variance 2. The average
power is P = P1 + P2, where P1 = δP and P2 = (1-δ)P. The capacity region for
the

T1

n
N(0,2)
T2

f2 y Receiver

Figure 3.7 The 2-link multiple access model.

two access channel [22] is given by

 δPf 2 
CMAC(T1  R)  Bln 1   nat/s,
 2σ 2B 
 
(3.19)
 δPf 4 
CMAC(T2  R)  Bln 1   nat/s.
 2σ 2B  δPf 4 
 

To achieve the capacity, the receiver first decodes x2, then subtracts x2 from
y and decodes x1.

As was observed in [23], the multiple access capacity region and the
broadcast capacity region can be shown to be equivalent when the sum of the
average power in the two transmitters is the same as the sum power in the
broadcast channel model. As a consequence, any point in the capacity region
for the broadcast channel can also be achieved in the multiple access channel

59
 Chapter 3-Cooperative Coding

capacity region. The proof is based on the transformation of power from the
access- to the broadcast domain.

For the time- and frequency-sharing, we refer to the first part of the section.
The results are the same, since for this situation, the links in tandem can be
considered to be point-to-point connections.

R(TR1)

C1
 Pf 2 
C1  Bln 1  nat/s,
 2σ 2B 
 

 Pf 4 
C2  Bln 1  nat/s,
 2σ 2B 
 

simple repeater (3.15)

(3.13) TS (3.17)
broadcast
(3.14) FS repeater + cooperative coding
(3.16)

0 C2 ½C1 Ccoop (3.16)

R(TR2)

Figure 3.8 Overview of transmission efficiencies

In Figure 3.8 we present the results for the different strategies in a general
way. FS and TS are the frequency- and the time-sharing regions,
respectively. We also indicate the performance for the degraded broadcast
channel together with the repeating strategies. Of course, the different points
have to be evaluated for particular link channel parameters and conditions.
We show that frequency-sharing is preferable over time-sharing. Time-
sharing also outperforms repeating for specific values of the link attenuation
factor f. From the presented material, it follows that there is an interesting

60
 Chapter 3-Cooperative Coding

connection between multi-user information theory and real communications,

such as communication over links in tandem.

3.4 Concluding remarks

This chapter originated from actual problems in power line communications.
The concept of the chapter is summarized in Figure 3.9.

transmission model

communications: capacity and coding

detect-and-forward
decode-and-forward
cooperative coding

multi-user information theory: capacity

time- and frequency-sharing

relay
broadcast
multiple-access

Figure 3.9 Structure of the material presented in Chapter 3

High attenuation hinders reliable transmission using power lines beyond 500
meters. Improving this number is the target of many investigations. Research
in this area started in 1994 with Olaf Hooijen1, a PhD student in the Institute
for Experimental Mathematics. We developed several models and described
the power line channel parameters. Measurements were carried out to
confirm the theory.

61
 Chapter 3-Cooperative
3 e Coding

A possibble solution to
t the attenuaation problem m is the appllication of reepeaters.
We conssider this poossibility and d investigatee “intelligentt” repeating: detect-
ward, decodee-and-forwarrd, and coopperative repeeating. We calculate
and-forw
the channnel capacityy for an artifficial connecction consistiing of severaal links.
For coopperative repeeating, we usse RS codes for an easy implementattion and
optimal efficiency reesults.

In the Seection 3.3 we

w translate th
he bus- or liink communnication conccept to a
multi-user informattion theory situation. The conceepts of “degraded
broadcasst” and “mullti-access” arre shown to be suited fo
or the descrip
ption of
two-linkk connectionss.

1
Olaf Hoooijen, Aspects of residential power line com
mmunications. Dissertation, University
U
Duisburg-Essen, 1998, IS
SBN 3-8265-34 429-8

62
 Chapter 4-Modulation and Detection

Chapter 4

Modulation and Detection

One of the popular topics in coding is the application of soft decision
decoding for additive white Gaussian noise channels. The reason for this is
the performance improvements in the decoding error rate that can be
obtained. We explain the idea of soft decision and then give examples of the
applications in combination with RS codes.

ei

si { + d/2, - d/2 } r i = si + ei

Figure 4.1 The additive white Gaussian noise channel

The second part of this chapter is concerned with non-Gaussian noise

channels. As an example, we assume that in addition to Gaussian noise, the
channel can have impulse noise, narrowband disturbances and fading. We
give a specific sub-class of RS codes that can be used to avoid degradation
due to these atypical disturbances.

63
 Modulation andd Detection
Chapter 4-M

4.1 Th
he Gaussiaan Chann
nel
The channnel model that we use is the Gaus sian channell as given in n Figure
4.1. Thee noise is addditive white Gaussian
G no ise (AWGN)) with the fo
ollowing
propertiees:

- the aaverage e i = 0;
- the pprobability density
d functiion is given bby

2
1 
p ei (  )  e 22 , (4.1.1)
2
2
 2  E (e i2 ) .

1
2ππ 2

p e i ( )
0

Figure 4.2 The

T Gaussian
n distributionn

We assuume that thhe transmittted messagee is binary,, with com mponents

represennted as si = +d/2
+ or si = –d
d/2, respectivvely. We exp
press the eneergy per
a (d/2)2. Th
informattion bit Eb as his is importtant if we want
w to comp pare un-
coded traansmission with
w coded trransmission.

64
 Chapter 4-M
Modulation andd Detection

threshold

α
-d/2 0 +d/2

Figgure 4.3 Coonditional pro

obability disttribution

The hardd decision deetector decid des for + or – and makes an error if fo
or si = +
T probabil ity of error is thus given by
d/2 the rreceived ri = si + ei < 0. The


 (  d   ) 2 / 2 2
PBSC.:= P( errror si = + d/2
d )=  1
22
e 2 d
0
 2 2
= 
1 e   / 2  d
2  2
d
2
 d2 
= Q  

2
 4 
d2 Eb
 
e 8 2 2
 e 2 ,

where w n Q(a) ≈ expp(-a2/2).

we used the appproximation

4.1.1 Errror correcction

a sn = (s1, s2, ..., sn),

For codeed transmissiion, the codee words are trransmitted as
where si {+ d’/2, - d’/2}.
d The en mbol is then defined
nergy per traansmitted sym
as Es = ((d’/2)2. For a fair comparrison, the am
mount of eneergy for the n coded

65
 Chapter 4-Modulation and Detection

symbols has to be equal to the amount of energy for k un-coded symbols, i.e.
nEs = kEb.

A code with minimum distance dmin can correct a maximum of t errors where
dmin > 2t. An indication of the behavior of the decoding error probability is
then given by

Pe, BSC  (PBSC ) t 1

d' 2 Es RE b
 (t 1)  (t 1)  (t 1)
2 2 2
 e 8σ  e 2σ  e 2σ .

We can observe, that the product R(t+1) determines the performance of the
coded system.

4.1.2 Error and erasure decoding

An (n,k) error correcting code with minimum distance dmin is able to decode
correctly, if for the number of erasures E and the number of errors t, the
following relation holds:

dmin ≥ E + 2t + 1.

The decision process is illustrated in Figure 4.4. The area where we declare
the detected symbols as erasure is denoted as E1 and E2.

E1 E2 E3

- d’/2 0 + d’/2

Figure 4.4 Decision regions for erasure and error decoding

66
 Chapter 4-Modulation and Detection

- The probability of an erasure is thus given by the probability that the

received symbol is in the region E1 or E2. This probability is determined
by the probability density function of the noise given the transmitted
symbol.
- The probability that an error occurs is equal to the probability that we
are in region E3 given that we transmit si = - d’/2. Hence, using Figure
4.4, we can say that for reliable decoding we have the condition that

dmin > n (P(E1 + E2) + 2P(E3)).

For hard decision decoding we make an error if the received signal is

positive given that we transmit si = - d’/2. The condition on correct decoding
then becomes

dmin > n (2P(E2) + 2P(E3)).

Since P(E1 + E2) > 2P(E2), we conclude that hard decision is the better
option.

4.1.3 Soft decoding

The Maximum Likelihood (ML) detector, searches for the code word that
maximizes L:= p(rn | sn), where

n
  (s i  ri ) 2 /2σ 2
1
L  p(r n | s n )  e i 1 .
(2π 2 ) n

This is equivalent to minimizing the squared Euclidean distance between rn

and sn, which is defined as

n
d2(sn, rn) =  (si  ri )2 .
i 1

If sn is a code word of length n, then the squared Euclidean distance between

cn, and sn is defined as

67
 Chapter 4-Modulation and Detection

d2(sn, sn) =  (s j  ŝ j ) 2  | D | d'2 ,

jD

where D is the set of positions where s j  ŝ j .

The probability of error, given sn is transmitted and rn = sn + en is received,

is given by:

P{ p(rnsn) < p(rnsn ) }

= P{ p(en = rn - sn ) < p(en = rn - sn ) },

where,

  (s i  ri ) 2 / 2 2
n 1
p(e ) = e D ,
2 D
(2 )

  (ŝ i  ri ) 2 / 2 2
n 1
p(e ) = e D .
2 D
(2 )

From p(en) < p(en) it follows, that a decision error occurs when

 2ri (ŝ i  s i ) / 2 2
eD  1,

which is equivalent to

 (si  ei )(si  ŝi )  0.

D

For linear codes, the distance spectrum (number of code words of a certain
weight) is the same for all code words. Therefore, we can calculate the
probability of error for the all-0 code word where all symbols are transmitted
as +d’/2. From this it follows that for si = - si a decision error occurs when

68
 Chapter 4-Modulation and Detection

d'
-| D |
2
  ei .
D

Now we use two properties of independent zero-mean Gaussian random

variables (GRV):

1. the sum of, zero-mean, independent GRV’s is again a GRV

with mean zero;
2. the variance of the sum is the sum of the variances.

From which,

E((  ei ) 2 )  | D | σ 2 .
D

The probability of error is thus given by

 d' 
 |D|   2 
Pe = Q 2   Q D(d' ) .
 | D | σ2   2 
   4σ 
 

For kEb = nEs and Es = d' / 2 we then have the un-coded bit error
probability

 Eb 
Pun-coded = Q , (4.1.2)
 2 
 

and the probability that we decode a specific code word at distance dmin
from the transmitted code word

   
 | D | k Eb   d min k E b   RE b d min
Pcoded = Q n  ≤ Q n   e 2 2 . (4.1.3)
 2    2 
   
   

69
 Chapter 4-Modulation and Detection

One can observe a gain in the decoding error probability when the product
Rdmin is larger than 1. If dmin is the minimum value of |D| for any pair of
code words, the union bound on the error probability for a binary (n,k) code
is given by
 
 d min k E b 
Punion  (2k)• Q n . (4.1.4)
 σ 2 
 
 
The problem is to find codes for which the net gain is positive. We define
the coding gain Gc in dB as

k
G c : 10 log10 d min dB. (4.1.5)
n

For practical codes, the product of the minimum distance and efficiency R
must be maximized to get the highest coding gain.

For binary or hard quantized decoding of an error correcting code, the

asymptotic decoding error probability is upper bounded by the probability
that we have more than dmin/2 errors for every code word, i.e.

d /2 RE b
 k E  min  (d min /2)
 b  k  e 2σ 2
Phard  2 k  Q n  2 . (4.1.6)
2 
 σ 
 

For soft decision decoding

 
 d min k E b  
RE b
d min
Psoft  2 k  Q n   2 k  e 2σ 2 . (4.1.7)
 σ 2 
 
 

From this, we see that soft decision brings a coding gain of 3 dB [30] (factor
of 2 in energy).

70
 Chapter 4-Modulation and Detection

1 0 ··· 1

parity
+

Figure 4.5 The Single Parity Check (SPC) code

Example The simplest code that we can find is the single parity check code
(SPC). For this code all code words have even Hamming weight and thus the
minimum distance is 2. Adding a parity bit to k = (n-1) information bits
gives the code word of length n and thus a code efficiency of (n-1)/n.

The asymptotic coding gain over the un-coded case is Gc = 10log102 = 3 dB.
In practice, the gain is less because we have to take the number of code
words into account. From simulation results, it follows that for n = 7 or 8, a
net coding gain of about 2 dB can be realized. If we fix the rate and increase
the code word length, the number of code words increases exponentially, and
the coding gain disappears.

Decoding for the SPC

To find the code word cn at minimum squared Euclidean distance from the
received word rn we act as follows:

1) take a hard (binary) decision on the received values. If the word is a

code word with even parity, stop and output the word as correct;
2) if the hard decision word is not a code word (odd parity), search for
the position with the smallest received absolute value, say rmin. Invert
the decision at this position.

Inversion of the hard decision value increases the squared Euclidean distance
with the smallest possible amount and the resulting hard decision word is a
code word.

These two steps are the basis for soft decision algorithms to be discussed
later. The SPC decoding is characterized by its extremely low complexity

71
 Chapter 4-Modulation and Detection

since, the decoder has simply to invert the least reliable bit (corresponding to
the smallest absolute demodulator output) whenever a parity error occurs in
the binary quantized received word.
Example Using the previous AWGN assumptions:

Encode +5 +5 -5 +5 -5 +5

Receive +4 +3 -4 -1 -3 +5

Hard decision 1 1 0 0 0 1

Correct 1 1 0 1 0 1

We change the decision at position four, since the hard decision at this
position is most likely to be wrong.

Remark If we compare the bit error rate for the coded situation using soft
decision and the un-coded situation using hard decision, we can expect a
gain of 3 dB. In theory, we can have a gain of 2 dB in channel capacity for
low signal to noise ratios (bad channels). Remark that the gain in channel
capacity is 2 dB at a decoding error probability approaching zero.

4.2 The Combination RS and SPC

We give several attempts to include “soft decision” decoding into the
decoding of RS codes. We do this by combining the SPC code with the RS
code, called concatenation. The SPC code has a minimum distance 2, and
thus the combination can in principle double the distance of the RS code.
This gain is the maximum that we can expect to achieve when using the RS
code only and performing “soft decision” RS decoding, which is considered
to be impossible due to its complexity and algebraic decoding structure. In
our combination, the 3dB gain is obtained by the SPC decoding at the cost of
a small rate loss.

4.2.1 RS symbols extended with a parity bit

We en- and decode in the following way:

Encoding Every symbol of the RS code is extended with a parity bit.

72
 Chapter 4-Modulation and Detection

Decoding The decoding steps are:

1. apply soft decision decoding to the symbols;
2. decode the symbols with the RS decoder.

Performance can be predicted from the results from SPC decoding.

parity bits of the symbols

···
RS code word symbols

Figure 4.6 RS code word combined with the SPC

4.2.2 Block wise RS-SPC coding

In this scheme, we encode and decode in the following way:

RS1 •••
RS2 1 2 3 •••

••• •••

RSN •••

1 2 ••• m RS code symbol 3 with m bits

Figure 4.7 Structure of the concatenated RS-SPC code

73
 Chapter 4-Modulation and Detection

Encoding We assume transmission of (N-1) RS code words from an (n,k)

code over GF(2m) followed by a word that is the bit wise modulo-2 sum of
the previous words. By linearity, this is also an RS code word.

Decoding For each received block of transmitted RS code words, the

decoder performs the following three major steps:

1. apply soft decision for each individual column (each column is an SPC
code word);
2. apply RS decoding for each row. If a detected, but uncorrectable error
pattern occurs, mark the row with a flag.
3. After receiving a block of N words Ri , i = 1,···, N, we calculate the word
En as the modulo-2 sum of the N rows, i.e.

N
E n :  Ri .
i 1
R i not flagged

Initially, En = 0n.
Four different situations can occur:
- no flag, En = 0n. No erasures and no detected errors. Continue with the
next block;
- no flag, En 0n. In this case we assume that there is only one RS code
word in error.
- one flag. Hence, one erasure occurs which can be solved by using the
fact that the modulo-2 sum of all the code words is 0n and thus the erased
RS word is equal to modulo-2 sum of the non-erased words;
Suppose that we define the initial block likelihood (equivalent to
Euclidean distance) as

N
L0   p(R k | RSk ) .
k 1

If only one code word is assumed to be in error, we have to add to one of

the decoded words the error word En. The best choice is the one that
maximizes the likelihood of the modified block. Hence, the decoder
determines the position that maximizes the corresponding likelihood, i.e.

74
 Chapter 4-Modulation and Detection

N
max  arg max p(R i | RSi  E n ) 
i 1,, N
 p(R k | RSk ) , (4.2.1)
k 1
k i

and gives as output RSmax + En instead of RSmax for the RS code word
according to eq. (4.2.1). The decoder thus calculates

N
L max  p(R max | RSmax  E n )   p(R k | RSk ) . (4.2.2)
k 1
k  max

Note that in (4.2.1) we only have to consider the locations in RSk where
En has nonzero components;
- more than one flag. More than one erasure occurs. Since the sum of all
RS code words is equal to 0n, we can calculate the modulo-2 sum of the
erased RS code words. For this block, given En, we can start again to do
the soft column decoding, followed by RS-row decoding. To avoid
looping, we put the condition that for the new block, the number of
erased RS code words is less than before. If not, decoding is stopped.

4.2.3 Performance

In this section, we discuss the performance of the different coding schemes.

A. We first consider RS-SPC (Section 4.2.1) decoding for an AWGN

channel with noise variance N0/2. The symbol error rate can be
approximated (4.1.3) as

 m  1  2RE b 
A Ps   Q ,
 2   N 0 / 2 

where R = × denotes the overall code rate, k/n is the code rate of
the (n,k) RS code over GF(2m) and Eb is the transmitted energy per
information bit.

B. For the concatenated RS-SPC coding scheme (Section 4.2.2) the symbol
error rate for high signal-to-noise ratios is approximately

75
 Chapter 4-Modulation and Detection

 4RE b 
B Ps  m( N  1)Q ,
 N 
 0 

with R = × . Since the symbol error rate for hard decision RS

decoding can be approximated by

 2kE 
 n b 
Ps  mQ ,
 N0
 

an asymptotic gain due to soft decision decoding of 10log10(2m/m+1))

dB (structure A) and 10log10(2(N-1)/N) dB (structure B) is expected: for
n = 31 (m=5) that is about 2.2 dB.

Since the minimum distance of the concatenated code (structure B) is twice

the minimum distance of the (n,k) RS code, an additional gain corresponding
to distance doubling can be expected. For exploiting this potential gain
erasure RS decoding as given by structure B is employed in addition to soft
decision SPC inner decoding.

The simulation results, for the different coding schemes that are based on the
use of antipodal signaling with coherent detection in a channel with additive
white Gaussian noise, are given in Figure 4.17. The RS code employed is the
short (31,21) code. For block structure B, N = 9 is chosen in the simulations;
in addition the maximum number of iterations is limited to 3. As can be
seen, symbol wise soft decision SPC decoding (curve 3) results in a gain
over the hard decision RS decoding of to about 1.5 dB at a bit error rate
(BER) of 10-5; this corresponds to an overall coding gain of about 4 dB. For
code structure B the best performance (curve 4) for BER < 10-3 is obtained.
Compared with the symbol wise soft decision SPC decoding, the gain is
about ½ dB at a BER = 10-5.

Remarks We applied the same technique to the concatenation of

convolutional codes and a single parity check code. We consider a
concatenated coding scheme that is characterized by a set of N multiplexed
convolutional codes forming the inner code and an outer (N,N-1) single-
parity-check (SPC) code. The inner convolutional codes are maximum-
likelihood (ML) decoded via N parallel, independently operating Viterbi
decoders. The decoded overall parity check sequence is used to correct
decoding errors resulting from Viterbi decoding. This outer soft decision

76
 Chapter 4-Modulation and Detection

SPC decoding can be realized exploiting the analogue demodulator output

used for Viterbi decoding before, with only a small amount of additional
hardware [31].

4.3 Non-Gaussian Disturbances

Soft decision decoding is based on the fact that the channel noise is Gaussian
distributed and additive. Furthermore, the channel parameters must be
known to the receiver, otherwise a mismatch will occur. In practice, channel
noise can be non-Gaussian and/or non-additive. Examples are narrowband-
and impulse noise. In addition, channel characteristics might be unknown to
the receiver, see [16,24]. We consider MFSK modulation combined with
coding as an efficient way to handle these disturbances.

4.3.1 MFSK modulation

MFSK modulation schemes [26,27] modulate symbols as one of the

sinusoidal waves described by

2E s
s i (t)  cos(2f i t); 0  t  Ts
Ts , (4.3.1)

where i = 1, 2, , M. The signal energy per modulated symbol is represented

by Es and

i 1
f i = f0 + , 1 i  M.
Ts

The signals are orthogonal and for non-coherent reception the frequencies
are spaced by 1/Ts Hz. Every Ts second we transmit a new symbol. We
restrict ourselves to the ideal MFSK modulation signaling as given in
(4.3.1). The bandwidth B required is approximately

M
B .
Ts

For an information rate of b bits per second, we have a symbol duration time

77
 Chapter 4-Modulation and Detection

log 2 M
Ts  ,
b

and thus a bandwidth efficiency of

b log 2 M
 bits/s/Hz.
B M

For large M, MFSK is spectrally inefficient.

MFSK modulation can provide for:

- a constant envelope modulation signal, to enable a transmitter's power

amplifier to operate at or near saturation levels;
- frequency spreading to avoid bad parts of the frequency spectrum;
- time spreading to facilitate correction of frequency disturbances and
impulse noise simultaneously.

cos2πfit

( )2
received y

( )2

sins2πfit sample at t = Ts

Figure 4.8 Basic envelope detector for frequency fi

4.3.2 Detection

Non-coherent MFSK detection is implemented by using a bank of 2M

correlators, with a quadrature pair for each frequency. For each quadrature
pair the output is added together using the square law to produce a metric for
the corresponding frequency candidate, see Figure 4.8. Non-coherent MFSK
detection measures received energy for M possible frequencies used. The M
detected envelopes can be used in the decoding process in several ways.

78
 Chapter 4-Modulation and Detection

Soft “envelope” detection

Normally, one chooses from a set of frequencies the one with the highest
energy present at a sampling instance Ts, assuming that all the M frequencies
are transmitted with energy Es. For a fair comparison with un-coded
transmission, we set Es/log2M = Eb, where Eb is the energy per information
bit. Furthermore, the Signal-to-Noise Ratio (SNR) for such a system is
defined as Es/N0 (refer to [26], p. 258).

The symbol error probability for transmission over an Additive White

Gaussian Noise (AWGN) channel with single sided noise power spectral
density N0, at high values of Es/N0, can be approximated as

Es

2N 0
Ps - uncoded  Me
(4.3.2)
Eb
log 2 M [ln 2  ]
2N 0
e ,

where the energy per bit Eb = Es/log2M. Note that we transmit log2M bits per
MFSK symbol. For AWGN channels the probability of a symbol error can
thus be made arbitrarily small by increasing M, provided that Eb/2N0 > ln2.
The cost for increasing M is the bandwidth required to transmit the signals.

In a coded situation, for an (n,k) code, where k = log2|C|, (4.3.2) becomes

log 2 |C| E b
 d min  
n 2N 0
Ps - coded  log | C | e . (4.3.3)

Example Suppose that we have an RS code with M-ary symbols, length M,

|C| = Mk and dmin = M – k + 1.Then, for large M, (4.3.3) is

Eb
MRlog 2 M[ln2  (1 R)  ]
2N 0
Ps - coded  e ,

where k/M = R. For (1 - R)·Eb/2N0 > ln2, the error rate goes to zero for
increasing M.

79
 Chapter 4-Modulation and Detection

A sub-optimum detector
As indicated before, spectral noise disturbs the demodulation process of M-
ary FSK. To overcome this problem, we give a possible structure for a sub-
optimum demodulator using a threshold after every envelope detector, under
the assumption of equal energy - and noise distribution for all M frequencies.

We modify the demodulator for our modulation/coding scheme as follows.

After every envelope detector we introduce a threshold T. For practical
reasons the value of T is set equal to E s /2 , where E s is the envelope
detector output for a transmitted frequency when no noise is present. For
noisy channels, we have to add a positive constant, which incorporates the
variance of the noise, to the threshold [24]. Envelopes that are larger than the
respective thresholds are given are marked with a symbol 1, otherwise we
assign the symbol 0. Thus, for a particular time instant, the output of the
envelope detector is a binary vector of length M.

We assume that we transmit M-ary code words of length n. These code

words can be represented in a binary matrix of dimension M x n, where
every column contains exactly one single symbol 1. In a hard decision
detector (detecting presence of a frequency), we therefore also put the binary
outputs in an M x n decoding matrix. We output the message corresponding
to the code word that has the maximum number of agreements with the
demodulator output.

The influence of the channel disturbances on the demodulator output can be

summarized as:

- narrowband noise causes large demodulator envelopes for particular

frequencies hit by the narrowband noise and thus may cause an envelope
detector output to be equal to 1 during many time instants. Narrowband
noise sources are emergency services, radio amateurs, etc. The
disturbance may be intentionally, or caused by an unknown source of
interference;

- impulse noise may put energy in larger parts of the spectrum in use for a
short period. Impulse noise can also have a periodic character. Impulse
noise results mainly from switching transients and is characterized by its
duration, amplitude and the inter-arrival time. We assume that impulse
noise causes all M outputs of the demodulator output vector to be equal
to 1;

80
 Chapter 4-Modulation and Detection

- the additive white Gaussian noise, also called the “background” noise,
introduces incorrect decisions, i.e., insertion or deletion of a symbol 1;

- in frequency selective fading, some frequencies may be in deep fade, or

experience high attenuation. As a consequence, the particular detector
output will be 0 for a long time (assuming this is the only disturbance).
In this case, an entire row of the decoding matrix is set to 0.

The effect of the different kinds of noise on the multi valued detector output
can be seen from Figure 4.9. We assume that M = 4 and transmit the code
word (1, 2, 3, 4) as frequencies (f1, f2, f3, f4).

1000 1010 1000

0100 0100 0000
0010 0010 0010
0001 0001 0001
no noise background noise

1111 1001 1000

0100 0101 0000
0010 0011 0010
0001 0001 0001
narrowband impulse fading

Figure 4.9 Several types of noise in the channel.

As one can see from Figure 4.9, row errors can be catastrophic for RS codes.
This can be explained as follows. Let EN denote the energy detected at the
demodulator due to a narrowband noise source. If EN exceeds the threshold
T, the demodulator will have a symbol that is “always on”. This poses a
problem, since the all-c vector (that is, the vector with all its symbols equal
to c, where c  GF(2m) is a valid code word for the RS code. Therefore, the
presence of narrowband noise disturbances will result in undetected errors.
We will discuss further consequences in the next section.

81
 Chapter 4-Modulation and Detection

4.4 COMBINED MODULATION and CODING

4.4.1 Permutation codes

Consider again an M-ary FSK signal set that consists of an orthogonal set of
M frequency-shifted signals that is used to encode information. We may
assume non-coherent demodulation and thus the bandwidth required is given
by B = M/Ts. For a code of cardinality |C| and code words of length n, the
number of information bits transmitted per second is given by

log 2 | C |
b bits/sec. ,
nTs

where Ts is the signal duration time. The transmission efficiency per second
per Herz is given by

b log 2 | C |
 bits/sec/Hz.
B nM

We consider a special class of codes, permutation codes, that is very well

suited for the noise problems mentioned before.

Definition A permutation code C consists of |C| code words of length M,

where every code word contains M different symbols.

Example M = 4, |C| = 4.

message transmit

1 ( 1, 2, 3, 4 )
2 ( 2, 1, 4, 3 )
3 ( 3, 4, 1, 2 )
4 ( 4, 3, 2, 1 )

As an example, message 3 is transmitted in time as the series of frequencies

(f3, f4, f1, f2). Note that the code C has 4 words with the property that any two
words always differ in 4 positions.

82
 Chapter 4-Modulation and Detection

TABLE 1

Code book for M = 4, dmin = 3

Message Code word Message Code word

1 (1, 2, 3, 4) 7 (4, 2, 1, 3)
2 (1, 3, 4, 2) 8 (4, 3, 2, 1)
3 (2, 1, 4, 3) 9 (1, 4, 2, 3)
4 (2, 4, 3, 1) 10 (2, 3, 1, 4)
5 (3, 1, 2, 4) 11 (3, 2, 4, 1)
6 (3, 4, 1, 2) 12 (4, 1, 3, 2)

TABLE 2

Two code books for M = 3

dmin = 2 dmin = 3

(1, 2, 3) (1, 2, 3)
(1, 3, 2)
(2, 1, 3) (2, 3, 1)
(2, 3, 1)
(3, 1, 2) (3, 1, 2)
(3, 2, 1)

TABLE 3

Code book sizes for M = 2, 3, 4, 5

M dmin =

2 3 4 5
2 2
3 6 3
4 24 12 4
5 120 60 20 5

The code as given in Table 1 has 12 words, each with M=4 different
numbers and a minimum difference between any two words or minimum
Hamming distance dmin of 3. For M = 3 we have the code books as given in

83
 Chapter 4-Modulation and Detection

Table 2. An interesting problem is the design of codes and the effect of

coding on the transmission efficiency. In Table 3 we give the code
construction results for M < 6. To maximize the efficiency, we have to find
the largest |C| for a given M and dmin. It is easy to see that for a code with
code words of length M each having M different numbers dmin  2. The
cardinality |C| of this code is M!. Hence, the bandwidth efficiency can be
defined as

b log 2 M! log 2 M
  ,
B M2 M

for large M. This is the same efficiency as un-coded M-ary FSK. The next
theorem gives an upper bound on the number of code words in a permutation
code.

Theorem For a permutation code of length M with M different code

symbols in every code word and minimum Hamming distance dmin, the code
cardinality is upper bounded by

M!
|C|  .
(d min  1)!

Proof Choose (M - dmin) different code symbols and put them at (M - dmin )
different positions. Then, with the remaining dmin symbols at the remaining
dmin positions, we can construct a maximum number of dmin code words at a
distance dmin apart. For every set of positions we can distribute the chosen
(M - dmin) different code symbols in (M-dmin)! different ways. We can choose

 M 
 
 M  d min 

different sets of positions. Hence, the overall product gives the maximum
number of code words with a minimum Hamming distance dmin ,

 M 
|C|  dmin   ( M - dmin )!
 M  d min 
M!
= .
(d min  1)!

84
 Chapter 4-Modulation and Detection

It can be shown mathematically, that for M = 6 and dmin = 5 the upper bound
cannot be met with equality. For dmin = 2, we always have equality for any
M. Blake [28] uses the concept of sharply k-transitive groups to define
permutation codes with distance M – k + 1. The structure of all sharply k-
transitive groups is known for k  2. In [28] it is also shown that the group
of even permutations on M letters is a permutation code with |C| = M!/2 code
words and minimum Hamming distance 3. To find good codes in general
appears to be quite difficult [29,74].

4.4.2 Permutation codes derived from RS codes

We put a constraint on the code words of an RS code over GF(2m), namely

that all symbols in a word must be different. This code will be a permutation
code. To construct a simple permutation code we use the first two rows of
the extended RS encoding matrix given by

1 1 1 1 … 1
GT = .
2 M-2
0 1   … 

The minimum distance of the code is M-1, where M = 2m is the length of the
extended code. If only the second row is used to generate code words, the
minimum distance is M-1. Including the first row, the last M-1 columns
generate a code with minimum distance M-2. Since the symbol at the first
position is non-zero, the overall minimum distance is (M-1).

Property Suppose that for the M-ary information pair (x,y) the information
symbol y ≠ 0. Then, the code words contain every symbol { 0, 1, , 2, …,
M-2 } only once. Every code word is a permutation of another code word.

This property follows from that fact that multiplication of the second row
with a nonzero element of GF(2m) realizes a permutation of the last (M-1)
symbols. Adding the same constant (first row) to all symbols realizes a
permutation over all M symbols.

The construction thus generates a permutation code with minimum distance

(M-1), and M(M-1) code words, which meets the upper bound with equality.
It furthermore allows the decoding using the regular RS decoding
algorithms. More constructions can for instance be found in [28,74].

85
 Chapter 4-Modulation and Detection

4.4.3 Performance for permutation codes

A message is encoded as a code word of length M with the integers 1,2, ,
M as symbols. The symbols of a code word are transmitted in time as the
corresponding frequencies.

Example For M = 4 and dmin= 4, a permanent disturbance (narrowband

noise) present at the sub-channel for frequency f4 and transmission of code
word {3, 4, 1, 2} could lead to a demodulator output {(3,4), (4), (1,4),
(2,4)}.

The decoder compares the demodulator output with all possible transmitted
code words. It outputs the code word for which the maximum number of
agreements with the symbols at the demodulator output occurs. For the
example, all symbols corresponding to code word 3 are present and thus
correct decoding follows. Since code words are different in at least dmin
positions, dmin – 1 errors of this type still allow correct decoding. The
example code has dmin = 4 and hence, we can tolerate the presence of 3
permanent disturbances in the demodulator output.

Example Suppose that we transmit the example code word {3, 4, 1, 2}. If
an impulse noise causes all envelopes to be present at three symbol
transmissions, then we may have as a demodulator output {(1,2,3,4),
(1,2,3,4), (1,2,3,4), (2)}.

Comparing this output with the possible transmitted code words gives a
difference (distance) of zero to the correct code word and one to all other
code words. Thus, even if three of these multi-valued outputs occur, we are
still able to find back the correct code word since there is always a remaining
symbol that gives a difference of one to the incorrect code words.

Example Background noise degrades performance by introducing unwanted

(called insertions) demodulator outputs or by causing the absence (called
deletion) of a transmitted frequency in the demodulator output.

For this type of “threshold” demodulation, the decoding is still correct for
dmin – 1 errors of the insertion/deletion type, since:

- The absence of a frequency in the demodulator output always reduces

the number of agreements between a transmitted code word and the
received code word by one. The same is true for the other code words

86
 Chapter 4-M
Modulation andd Detection

haviing the sam me symbol at a the samee position. IfI the symb
bols are
diffeerent, the num
mber of agreeements doess not change;;

- the appearance of every un nwanted ouutput symboll may increease the

nummber of agreeements betw
ween a wronng code worrd and the received
r
codee word by one.
o It doess not decreaase the num mber of agreeements
betw
ween a transm
mitted code word
w and thee received code word.

Example In frequenncy selectivee fading the symbol eneergy disappeears and

thus thee symbol 0 will be gen nerated by thhe hard deccision detecttor. The
symbol disappears for
f the correect code worrd as well ass for the com mpeting
code woords. Correct decoding is possible if lless than dmiin symbols arre set to
0.

In concllusion, we can
c say that the introducced thresholdds in the modified
m
demodullator in commbination wiith a code aallow the correction of dmin – 1
incorrectt demodulaator outputss caused by narrowb band- , im mpulse-,
backgrouund noise or frequency seelective fadinng.

Figure 4.10 General con

F ncatenation fr
from Forney [33]

Conclussion We desscribe a mo odulation/codding schemee that is cap pable of

handlingg frequency disturbances
d like narrowbband noise or
o impulse no oise. For
this we use the concept of MFS SK modulatiion combineed with perm mutation
codes. W
We modify thhe “simple” non-coherent
n t demodulato
or in such a way
w that
the dem
modulator outtput becomees multi-valuued. The dem modulator output is
used inn the decoding of the permuutation code. The deescribed
modulatiion/coding scheme
s leadss to the impllementation of an overall robust
system. Applicationns of permu utation codess can be fo ound in pow wer line
communnications [88].

87
 Chapter 4-Modulation and Detection

4.5 Other Code Combinations

The original idea of code concatenation can be found in the work of David
Forney [33]. The raw channel together with an inner decoder behaves like a
super channel, see Figure 4.10. It turns the raw channel into another channel,
the super channel that is suited for the decoding process of the outer coder.
The goal of concatenation is thus to make the coding schemes suitable for
the particular channel conditions.

Information

RS decoding

independent errors

de-interleaver 1

decoding errors

convolutional
decoder
independent noise

de-interleaver 2

noise with memory

digital channel

data from the encoding operations

Figure 4.11 General concatenated coding scheme

88
 Chapter 4-Modulation and Detection

4.5.1 RS-Convolutional code combination

In Figure 4.11 we give a concatenation scheme with an RS outer code and a

convolutional inner code. The de-interleaving is supposed to change an
arbitrary channel noise distribution into a distribution that looks completely
“random” to the decoder. In this way, only average error probabilities are of
importance, not the distribution itself. In this way, complicated channel
modeling can be avoided. What remains is the design of the interleaving
procedure.

Information

RS encoding

8-bit symbols

interleaver

8-bit symbols

constrained
encoder
17 bits RLL

to the recording medium

Figure 4.12 Concatenation of RS with RLL

The digital channel delivers data for the convolutional decoder. The memory
in the data can be removed by the de-interleaver 2, such that the
convolutional decoder receives independent symbols. Convolutional
decoders are sensitive to dependencies between noise symbols. The
convolutional decoder, in general, produces decoding errors that have a burst
nature. We collect the output in symbols to be processed by the RS
decoding. To be able to decode long bursts of errors, we spread the symbols
over different RS decoders. In this way, simple low complexity parallel RS
decoders, each decoding a small number of errors, can handle bursts of
symbol errors.

89
 Chapter 4-Modulation and Detection

If the channel has AWGN, we do not need de-interleaver 2. The

convolutional decoder does the soft decision decoding, whereas the RS
decoding takes care of the burst decoding errors.

4.5.1 RS-constrained code

In recording, we have to generate constrained sequences that can be stored

into or written onto a medium. Here, RS codes also play an important role.
For disk applications, very often an RS code over GF(28) is used, because
the 8-bit symbols can be translated in an efficient way into, for instance,
Run Length Limited (RLL) blocks, see [32]. An RLL block puts constraints
on the runs of bits. There can be a minimum and a maximum run of the same
type of bits for synchronization or control purposes. An encoding scheme is
given in Figure 4.12. The idea is that at reading, errors in the RLL sequence
produce symbol errors after mapping back to the 8-bit symbols. The RS code
can also be combined with other constrained codes, like equal weight codes.

4.5.2 RS Product code

A classical combination of codes is that of a product code. The idea,

generated by Peter Elias [34], of a product code is given in Figure 4.13,
where code words are formed by n rows and N columns. A column of kr
information symbols is converted into a column of n code symbols. Then,
every row with kc symbols is converted into a row with N symbols. We end
with a n × N matrix. The row and/or the column code can be an RS code.

kc
C1 C2 ••• CN
c11 c21 ••• cN1 row 1
c12 c22 ••• cN2 row 2
kr
••• ••• •••

c1n c2n ••• cNn row n

Figure 4.13 A product code with n rows and N columns

The minimum distance of the product code is larger than or equal to the
product of the distances of the individual codes. This can be seen as follows.

90
 Chapter 4-M
Modulation andd Detection

Any sym mbol of a nonnzero column n word of miinimum weig ght dc gives rise to a
row codde word withh minimum weight
w dr annd thus the minimum
m weeight of
product code word is i at least dc × dr. Since low weight column codee words
do not nnecessarily giive low weigght row codee words, we have a lower bound
on the mminimum disstance. We can see this pproduct codee as a generaalization
of the coombination RS-SPC
R discu
ussed earlierr.

Remark k A row deccoding error occurs

o if at lleast dr/2 colu
umn errors occur.
o A
column error occurs if at least dc/2 errors oc cur in a colu umn. Hence, we can
guaranteee correct deecoding up to dc × dr/4 errrors.

Figure 4.14 RS product code for DvD

F D application
ns

In Figure 4.14 we giive the RS prroduct code used in DvD

D. The data symbols
s
are conttained in 1722 columns and
a 192 row ws. Each of the 172 coluumns is

91
 Chapter 4-Modulation and Detection

extended with 16 parity symbols which give a total of 208 rows. Every row
is extended with 10 parity symbols which give a total of 182 columns. This
procedure gives an RS product code with 208 rows and 182 columns.

If we do a simple row and column decoding, the combination can correct 8

column errors and 5 row errors. By using iterative decoding where row and
column decoding is repeated, performance improvements can be obtained.
In Figure 4.15 we give a general concept for such an approach (source:
Scholarpedia, G.D. Forney)

outer inner
interleaver
encoder encoder

channel

outer inner
de-interleaver
decoder decoder

interleaver

Figure 4.15 Concept of iterative decoding of concatenated codes

4.6 Concluding remarks

Soft decision decoding based on additive white Gaussian noise promises an
asymptotic 3 dB gain in the decoding error probability. This is equivalent to
a distance doubling using the regular hard decision decoding. Therefore, it is
interesting to see how this gain can be realized. The gain for a simple R =
7/8 single parity check code is about 2 dB1. About the same gain can be
expected for a concatenated SPC-RS coding scheme.
1
Peter Foerster, Soft Decision Decoding, PhD University of Darmstadt

92
 Chapter 4-Modulation and Detection

The problem of the soft decision decoding is the knowledge of the noise
source. If the noise is pure AWGN, then the soft decision metrics are correct.
However, in practical systems this knowledge is not often available and thus
the advantages of soft decision decoding disappear or soft decision decoding
even makes the performance worse.

RS codes are symbol oriented codes and thus, in principle the transmission
over an AWGN channel with independent transmissions does not exploit the
power of an RS code. Therefore, concatenation with a short high rate inner
code that performs the soft decision decoding or that decodes single errors is
preferred. The RS code can act as an outer code that corrects the symbol
errors caused by decoding errors from the inner code or caused by channel
symbol errors.

In the second part of the chapter we consider non-Gaussian noise. Practical

communication systems using wireless or power line communications are
disturbed by impulse- or narrowband noise. We chose diversity in time and
frequency to facilitate correct decoding. As a modulation scheme we use
MFSK for frequency spreading. The permutation code performs the time
spreading. We show that several types of noise can be corrected using a
simple detection scheme. We give a simple (n-1,n) RS code that can be seen
as a permutation code with dmin = n-1. Further constructions using trellis
codes can be found in [95].

We summarize the content of this chapter in Figure 4.16.

AWGN soft decision SPC or SPC-RS

non-Gaussian time/frequency permutation

noise diversity coding

Figure 4.16 Overview of the content of Chapter 4

93
 Chapter 4-Modulation and Detection

100

10-1

10-2

10-3

10-4

10-5 (1)

10-6 (3) (2)

(4)
10-7
0 2 4 6 8 10 dB
10 log10(Eb/No)

Figure 4.17 Bit error rate versus signal-to-noise ratio for antipodal
signaling in AWGN channels: (1) un-coded; (2) hard decision
(31,21) - RS decoding; (3) concatenated(31,21) - RS - (6,4) SPC soft
decoding as in structure A, R = 0.56; (4) concatenated (9,8) SPC,
(31,21) - RS decoding, R = 0.6

94
 Chapter 5-The Wiretap Channel

Chapter 5

The Wiretap Channel

A classical problem in data transmission is that of wiretapping. The wiretap
channel was introduced by Wyner [35], see Figure 5.0. The idea is that a
wiretapper can listen to the normal communication between two parties. The
encoder’s objective is to maximize the amount of information transmitted to
the legal receiver, while keeping the wiretapper as uninformed as possible
[39,40]. Another option is the transmission of a secret to the legal receiver
without giving any information to the wiretapper. One of the interesting
consequences of the developed theory is that there is a close connection
between wiretap communication and the problem of secure biometric
authentication - and verification schemes, see also Chapter 6.

In the wiretap communication model, the message sk is encoded as xn,

transmitted via a noisy channel and received as yn by the legal receiver. The
wiretapper also receives a disturbed word zn.

encoder decoder (legal)

message
channel
sk xn yn sk

channel zn
wiretapper

Figure 5.0 Classical wiretap channel model

95
 Chapter 5-The Wiretap Channel

Practical encoding schemes are to be designed that achieve the goal of

privacy and transmission efficiency.

5.1 Wiretap Channel Models

We consider two basic communication models each with a different position
for the wiretapper, see Figure 5.1 and Figure 5.2. We only deal with discrete
wiretap channels, and we assume that the transmission is binary, using
Binary Symmetric Channels (BSC) with cross over- or transition

encoder BSC (p) decoder (legal)

message

sk xn yn sk
n
e
bn BSC (q)

zn
wiretapper

Figure 5.1 Binary wiretap channel model A

encoder BSC(p) decoder (legal)

message

sk xn yn sk
en

zn
wiretapper
BSC (q') bn

Figure 5.2 Binary wiretap channel model B

96
 Chapter 5-The Wiretap Channel

probabilities p, q’ and q. For p < q’, p < ½, the configuration B can be

converted into configuration A by substituting q = (q’- p)/(1 - 2p), see Figure
5.3.

A message sk of length k bits is transmitted as a code word xn of length n.

The wiretapper, receiving zn, tries to estimate the selected input message sk.

The first problem is the maximization of the amount of transmitted

information to the receiver, I(Sk;Yn) over the main channel from X to Y,
while at the same time minimizing the amount of information, I(Sk;Zn),
leaked to the wiretapper. As usual in cryptography, the wiretapper knows the
encoding scheme for the transmitter and the decoding scheme used at the
receiver.

0 0 0 0

p q ≡ q'

1 1 1 1

Figure 5.3 Transformation of two “tandem” channels

In information theory, we express the uncertainty about the message sk given

the received vector zn by the equivocation H(Sk | Zn), i.e.

H(Sk | Zn) = H(Sk) - I(Sk;Zn). (5.1)

The information sk is encoded as a code word xn and we assume that this

code word is transmitted to the legal receiver at an efficiency I(Xn;Yn). The
mutual information I(Sk;Zn) = I(Xn;Zn) when we assume that every sk
uniquely determines xn. Hence, (5.1) can be written as

H(Sk | Zn) = I(Xn;Yn) - I(Xn;Zn). (5.2)

The goal of the transmitter is thus to maximize H(Sk | Zn). It has been shown,
that for a binary symmetric main - and a binary symmetric wiretap channel
the maximum is given by the secrecy capacity Cs,

97
 Chapter 5-The Wiretap Channel

Cs  
max I(X n ; Y n ) - I(X n ; Z n ) 
P(X n )

(5.3)
 max I(X n ; Y n )  max I(X n ; Z n )
P(X n ) P(X n )

 n C main  C wt ,

where we assume that the main channel is less noisy than the wiretap, i.e.
I(Xn;Yn) ≥ I(Xn;Zn) for all input distributions P(Xn). We can reformulate
(5.3) as

R    C main  C wt ,

H (S k )
R : , (5.4)
n

H (S k | Z n )
 : .
H (S k )

Figure 5.4 gives a graphical illustration of (5.3).

1


Cs Cmain
R

Figure 5.4 Information rate R versus equivocation 

98
 Chapter 5-The Wiretap Channel

The secrecy capacity Cs is thus the maximum difference between the

information transmitted to the legal receiver and the information transmitted
to the wiretapper.

For binary symmetric channels, the normalized capacity for the main
channel with transition probability p is Cmain = 1- h(p) bit/transmission,
where h(p) is the binary entropy function. For the wiretap channel the
normalized capacity is Cwt = 1 – h(q) bit/transmission. Hence, according to
the definition, the normalized secrecy capacity

Cs = h(q) – h(p) bits/transmission. (5.5)

The interpretation is, that the wiretapper, after observing a block of n

received symbols, has an uncertainty of n(h(q) - h(p)) bits for the transmitted
message.

In Chapter 6, we describe the concept of biometric authentication. This

concept can be modeled as a wiretap channel. Performance improvements
can be obtained by using the fact that the legal receiver can also observe the
data received by the wiretapper. The legal receiver thus has the availability
of two parallel channels: from X to Y and from X to Z, see also Figure 6.5.
We therefore reformulate the secrecy capacity for the binary symmetric
channels as

C s  :  max [I(Xn ; Z n Y n )  I(X n ; Z n )]

P(Xn )

 max [I(Xn ; Y n )  I(X n ; Z n | Y n )  I(X n ; Z n )]

P(Xn )

 
 max I(X n ; Y n )  I(X n ; Z n )  H(Zn | Y n )  H(Zn | X n Y n ) ,
P(Xn )

which enlarges the secrecy capacity as given in (5.3), since conditioning

makes the entropy smaller.

99
 Chapter 5-The Wiretap Channel

5.2 Noiseless Main Channel

We first consider the noiseless main channel, where p = 0 and thus the
secrecy capacity is h(q) bits per transmission.

n-1 message bits

1 0 ··· 1

Parity


Secret bit

Figure 5.5 One bit secrecy using the Single Parity Check code

5.2.1 Single Parity Check code (SPC)

For this channel a simple strategy that allows transmission of one secret bit
to the legal or main receiver can be explained as follows. The transmitter
uses a single parity check code of length n with n-1 information bits and one
parity bit. The transmitter adds his secret bit modulo-2 to the parity, see
Figure 5.5.

The wiretapper can try to recover the secret bit by adding up, modulo-2, the
received n bits. The probability that the secret bit is incorrect is the
probability that an odd number of errors occurred during the transmission of
the n bits, since this changes the parity of the word, i.e.

Pe = 1 – ½ (1+(1-2p)n),

which converges quickly to ½ for appropriate values of n. For Pe = ½, the

wiretapper has a probability of ½ of being correct. This is equivalent to an
entropy of one bit, which is exactly the purpose of the coding.

100
 Chapter 5-The Wiretap Channel

Note We transmitted (n-1) message bits and 1 secret bit to the legal
receiver. The wiretapper only has a corrupted message (errors occur with
probability p) .

5.2.2 Hamming code

Before we discuss the application of RS codes, we give an example using

the Hamming code.

For the (7,4) Hamming code, we choose the encoding matrix G4,7 and the
syndrome former HT as given by

1000111 111
110
0100110 101
G4,7 = , HT = 0 1 1 .
0010101 100
010
0001011 001

A message m4 is encoded with G4,7 and a secret of three bits, s3, is added to
the last bits of the code words. Since the legal receiver has a noiseless
channel he can recover the message and secret without any decoding errors.

The wiretapper obtains

r7 = m4G4,7 + (04, s3) + e7.

To recover the secret, the wiretapper may calculate the syndrome

z3 = r7 HT

= e7HT + s3 .

After calculating the syndrome at the receiver, we estimate the secret s 3

equal to the syndrome z3. This is equivalent to the estimation of the most
likely noise sequence e7 = 07, the all-zero sequence. The probability of error
is then equal to the probability that a noise sequence occurs that is not equal
to a code word, since this would give the incorrect estimate. Hence,

Pe = Probability(e7 HT  03)

101
 Chapter 5-The Wiretap Channel

1  A i pi (1  p) n i , (5.6)
C

where Ai is the number of code words of weight i. For the Hamming code
the weight distribution is known and can be used to estimate the error rate.
For the sixteen possible code words, we have A0 = 1, A3 = A4 = 7, and A7 =
1. A first approximation for Pe gives Pe ≈ 1 – (1 - p)7.

Note The wiretapper can of course use another strategy that gives him a
lower error rate.

In general, for linear codes, we encode the information mk and secret sn-k as

xn = (mk, sn-k) · Ik T
,
0 In-k

where we use the systematic encoding matrix Gk,n = [ Ik, T]. We continue
with RS codes using the general encoding method.

5.2.3 Reed-Solomon codes

We investigate the more complicated situation where we transmit symbols.

Suppose that we use a systematic symbol error correcting RS code over

GF(2m) with a redundancy of (n-k) symbols. The equally probable messages
mk are encoded as mk Gk,n, where Gk,n is the systematic encoding matrix of an
RS code. The schematic representation is given in Figure 5.6. We add to the
(n-k) check symbols the equally probable secrets of length (n-k) symbols,
represented as (0k, sn-k).

During transmission, we assume that a symbol transmission error occurs

with probability p < ½. At the receiver side we thus obtain

rn = mkGk,n + (0k, sn-k) + en.

102
 Chapter 5-The Wiretap Channel

where en and 0k are the noise- and the all-zero symbol sequence,
respectively. As before, the wiretapper calculates the syndrome

zn-k = rn HT = enHT + sn-k.

For equally probable secrets sn-k, every syndrome is connected to a secret and
thus a decodable noise sequence. The minimum error probability receiver
would chose the secret equal to the syndrome, for this connects the secret
with the most likely noise sequence, the all-0 sequence. The probability of a
decision error can be estimated as the probability that a noise sequence
occurs that is not equal to a code word, i.e.

Pe  1   A i pi (1  p) n i
C

 1 - (1- p)n - ((2m)k -1) · pdmin·(1-p)n - dmin, (5.7)

where we lower bounded the error probability by assuming that all non-zero
code words have minimum weight dmin. The bound (5.7) can be close to 1
for large values of n, but needs precise evaluation for specific values of k
and n. The uncertainty that remains at the wiretapper is then equal to (n-k)
symbols for equally probable messages.

5.3 Noisy Main Channel

For the noisy main channel a more complicated situation occurs. We modify
the strategy as given in section 5.2. The goal is to transmit a message and a
secret to the legal receiver over a noisy main channel, while keeping the
wiretapper uninformed about the secret.

Suppose that the maximum fraction of channel symbol errors for the main
channel is p < ½ and that for the wiretap channel the symbol error fraction is
q > p. In the encoding procedure, we use an RS code with a redundancy n –
k = 2pn.

103
 Chapter 5-The Wiretap Channel

k message symbols n-k “check” symbols

···

n-k secret symbols

Figure 5.6 Encoding for (n-k) secrecy symbols.

To achieve the goal, we use the following RS encoding matrix

representation. The standard encoding matrix Gk,n for an RS code as given
in (A4.1) with dimension k × n, can be brought into semi-systematic form as
follows.

Step 1. Consider the first u rows and bring this part of the matrix Gk,n in
systematic form using only the first u rows. The sub-matrix formed by the
top u rows in called Gu,n. The matrix Gu,n is an RS encoding matrix with
minimum distance (n – u + 1). With the systematic part we can create a zero
sub-matrix in the bottom part of Gk,n. The resulting matrix Gk,n is given by

u k-u n-k

Iu A B u
Gk,n = .
0 C D k-u

Gk,n generates an RS code with minimum distance (n-k+1) and therefore, the
bottom part must have rank equal to v = (k - u). If not, the minimum distance
cannot be equal to (n-k+1). Hence, by using row and column operations we
arrive at step 2.

104
 Chapter 5-The Wiretap Channel

Step 2. The bottom v rows of the matrix can be manipulated in such a way
that Gk,n is equivalent to

u k-u n-k

Gu,n Iu T U u
Gk,n ≡ Gs-sys := =
0 Gv,n-u 0 Iv W v= k-u

Property The RS codes generated by the matrix Gs-sys, Gu,n and Gv,n-u have
minimum distance (n-k+1), (n-u+1) and (n-k+1), respectively.

The transmitted word cn for the information word mu and the secret word sv
of length u and v, respectively, is given by

cn = (mu , sv) • Iu T U

0 Iv W

= (mu, mu T + sv, mu U + sv W ).

The word cn is transmitted and corrupted by the noise word e1n in the main
channel and by the noise word e2n in the wiretap channel. When the number
of errors in the main channel is less than (n-k)/2, where k = u + v, these
errors are correctable and the legal decoder can recover mu and sv.

The wiretapper has a channel that is worse than the main channel, and thus is
assumed not to be able to decode the message. However, the wiretapper may
try to imitate the legal receiver and calculate a syndrome, using the
syndrome former HT for Gu,n, as

zn-u = (cn + e2n) • HT

= [(mu, mu T + sv, mu U + sv W) + e2n ] • T U+TW

Iv W

0 In-k

105
 Chapter 5-The Wiretap Channel

= (sv, 0 ) + e2n • HT .

To minimize the error probability, the receiver would choose the secret equal
to the first v symbols of the syndrome, for this connects the secret with the
most likely noise sequence, the all-0 sequence. The probability of a decision
error can be estimated as the probability that a noise sequence occurs that is
not equal to a code word and is given in (5.7). For Pe→1, the uncertainty that
remains at the wiretapper is then equal to v = k – u = (2q – 2p)n symbols.

The wiretapper may try to profit from the fact that the number of correctable
error patterns is less than the number of possible syndromes. However, for
code word symbols from GF(2m), code word length n = 2m - 1, and a
redundancy 2qn = n – k + v = n - u, the number of correctable error patterns
can be estimated as

qn
n
| e2n |   (2m ) t  t   (n  1) 2qn  (n  1) n  u . (5.8)
t 0

Hence, every possible syndrome pattern corresponds with a correctable error

patterns. In fact, for every different sv a different decodable error pattern can
be given. Given sv and the error pattern, a unique mu can be determined.
Since the sv word is of length v, we say that the wiretapper has v symbols of
uncertainty, where v = k – u = 2q – 2p. This is the same result as given in
the Gelfand-Pinsker [36] bound for the wiretap channel.

1 1 1 1 1 1 1
G= 1  2 3 4 5 6
1 2 4 6  3 5

Figure 5.10a RS encoding matrix for n = 7 k = 3

Example We conclude with two simple examples of codes, based on the

minimum distance 5, two symbol error correcting RS code for GF(23)
generated by the minimal polynomial 1 + X2 + X3, see appendix A2, Figures
5.10a, 5.10b, and 5.10c.

106
 Chapter 5-The Wiretap Channel

1 0  6 5 2 4
G= 0 1 5 4  3 6
0 0 1 4 1 6 4

Figure 5.10b Reed-Solomon encoding matrix for one symbol security

1 1 1 1 1 1 1
G= 0 1 0 5 1  
0 0 1 4 1 6 4

Figure 5.10c Reed-Solomon encoding matrix for two symbol security

5.4 Noiseless Main Channel and Partly Observed

Message
Suppose that code words are transmitted over a wiretap channel of type II
[36], where the main channel is noiseless, xn = yn, and the wiretapper can
examine a subset  of size- of the code word symbols of his choice.
Ozarow and Wyner [36] examined this situation and defined the
equivocation for the wiretapper as

Δ  min H(Sk | Zμ ).
τ:μ

The concept of equivocation is equivalent to the average number of source

symbols that cannot be found given  code word symbols. The minimization
in the definition of the equivocation reflects the best possible situation for
the wiretapper. A system designer tries to find a coding scheme that provides
an optimal tradeoff between the rate of transmission and the equivocation.

We will discuss the properties for the case when we transmit symbols from
an RS code over GF(q = 2m). To explain the coding strategy, we make use of
a systematic RS code with dimensions k × n. The four encoding/decoding
steps are:

107
 Chapter 5-The Wiretap Channel

1. randomly generate a code word;

2. add the secret of length n-k symbol wise to the last n-k code
symbols;
3. the legal receiver immediately retrieves the secret by subtracting the
code word, identified by the first k symbols, from the received word;
4. the wiretapper tries to find the secret based on a subset  of size-
of the code word symbols.

Performance can be analyzed using the following observation: Let i and j, 0

≤ i+j = μ ≤ n, be the number of symbols in yn known to the wiretapper from
the first k and the last n-k symbols (check part), respectively. We distinguish
the following two cases.

encoder decoder (legal)

message

sk xn yn ŝ k

zμ
wiretapper

Figure 5.7 The general wiretap channel II

Case 1: μ ≤ k (Figure 5.8a)

For the RS encoding matrix the rank of any k × k sub-matrix is equal to k,

and thus there are qj possible different segments at the j positions, specified
by the wiretapper in the check part. Every particular segment, together with
the observation, corresponds to j secret symbols. Since there are qj possible
different segments, the wire- tapper does not gain any information about the
transmitted secret by observing the j symbols. For equally probable code
words the total equivocation, or logarithm (base q) of the number of possible
segments, is thus 1 = (n-k) symbols. We visualize case 1 in Figure 5.8a.

108
 Chapter 5-The Wiretap Channel

Case 2: μ > k (Figure 5.8b)

The wire tapper chooses j positions in the check part. For k < μ = i + j ≤ n,
we have j > k – i. Since k positions uniquely determine the code word there
are j – k + i positions with symbol values determined by the specific code
word. The total number of unknown symbols, equivocation, for the wire
tapper is thus (k – i) + (n – k) – j = n – (i + j) = n – μ. We visualize case 2 in
Figure 5.8b.

k n-k

Ik

i j

Figure 5.8a Selection of i + j columns in G, where i + j ≤ k

k n-k

Ik

k-i
i j

Figure 5.8b Selection of i + j columns in G, where k < i + j ≤ n

For μ ≤ k, the equivocation is (n - k) symbols. For μ = n, the equivocation

is zero symbols. For μ > k the equivocation is n – μ. We summarize the
results in the “minimum possible equivocation” region as given in Figure 5.9
which was derived by Ozarow and Wyner. We conclude that RS codes
perform according to the bound!

109
 Chapter 5-The Wiretap Channel

Example We use the RS code of length 3 over GF(22). The encoding matrix
looks like

1 0 α
G2,3 = .
0 1 α2

The word

y3 = x2 • G2,3 + (0, 0, s1),

where s1 is the secret symbol to be protected. It is easy to verify, that the

possible observation of μ = 1 symbol by the wiretapper does not reveal the
secret symbol.

n-k

k n μ

Figure 5.9 The capacity region for μ observations by the wiretapper

For general linear block codes, the equivocation depends on the rank of the
selected sub-matrix. In [37], Forney introduced the concept of Inverse
Dimension/Length Profile (IDLP) of an (n,k) linear block code C, which
demonstrates the dependency of the equivocation on the structure of the
matrix G generating the code. The IDLP of C is defined as a sequence of
length n + 1 with components

k μ (C)  min  rank Pτ (C) , 0    n ,

τ

110
 Chapter 5-The Wiretap Channel

where P (C) is the selected sub matrix defined by the μ positions. For our
example, k(C) = {0, 1, 2}. An upper bound on IDLP is given by Forney in
[37] as

k, k≤μ≤n
kμ(C) = .
μ, 0≤μ≤k

The RS codes are codes that meet the upper bound. In the situation where the
wiretapper has additional information on the encoded data, an extension of
the IDLP has been defined in [38].

5.5 Concluding Remarks

We describe the classical wiretap channel model with a legal receiver and a
wiretapper. For this model we derive the secrecy capacity for the noiseless
main channel and for the noisy main channel. We give a coding method
using RS codes and compare the performance with the optimum capacity
results. As a special case, we explain the idea of a wiretapper observing
only a specific number of symbols from the transmitted message to the legal
receiver. Here we also conclude that RS codes obtain optimum performance.
The content is given in Figure 5.10.

Wiretap channel models

Secrecy capacity

Coding for noiseless main channel

Coding for noisy main channel

Noiseless main channel and partly observed messages

Figure 5.10 Overview of the content of Chapter 5

111
 Chapter 5-The Wiretap Channel

An interesting result was obtained by Bin Dai in [89], where he considered

the wiretap channel with noiseless feedback. Feedback can increase the
secrecy capacity region! Further basic research in this area has been carried
out by Chai Mitrpant1, Yanling Chen2, and Bin Dai3. They showed that side
information available at the transmitter enlarges the secrecy capacity region
in the Gaussian case, whilst in the discrete case the problem is still open.
1
Chai Mitrpant, PhD, University Duisburg-Essen, Information Hiding, 2003
2
Yanling Chen, PhD, University Duisburg-Essen, Wiretap Channel With Side Information,
2007
3
Bin Dai, PhD, Jiao Tong University, Shanghai, 2013

112
 Chapter 6-Biometrics

Chapter 6

Biometrics
In Shannon’s cryptography landmark paper [50], he defines a cryptosystem
as depicted in Figure 6.1. A message m is encrypted with a private key into a
cryptogram c. At decryption we use the same key and reconstruct the
message m. A cryptosystem is said to have “perfect secrecy” when for every
message m and ciphertext c (cryptogram) the probability P(m | c) = P(m). In
this case, it can be shown that the entropy H(M) = H(M | C). In all other
cases, the difference H(M) – H(M | C) is positive and we call the difference
the leakage of a system.

key K key K

cryptogram C
encrypt decrypt
e(M,K)=C d(C,K)=M

message M message M

Figure 6.1 Schematic representation of the Shannon cipher model

Without loss of too much generality, we assume that the message M and
cryptogram C are connected via one unique key K. Then, the condition for
“perfect security” can be given as

113
 Chapter 6-Biometrics

H(M) = H(M | C) = H(M,C | C) = H(K | C) ≤ H(K).

The interpretation is that the minimum average description length of the

message must be smaller than that for the key in order to have “perfect
secrecy”.

In the following situation, we assume that the receiver has a noisy version of
the key, see Figure 6.2.

noise

key K key K

cryptogram c
encrypt decrypt
e(M,K)=C d(C, K)=M

message M message M

Figure 6.2 Schematic representation of the noisy cipher system

Again, we assume that a message M and cryptogram C together also

uniquely determine the key K. In addition, we expect that for the noisy key
K and C uniquely determine M. Hence, there is no decryption error. For
“perfect secrecy”, we have the relation H(M) = H(M | C) and thus for (M,C)
→ K and (K,C) → M, we have

H(M | C) = H(M,C | C) = H(K | C)

= H(M,K | C) – H(K | M,C)

= H(K | C) + H(M | K,C) – H(K | K)

= H(K | C) – H(K | K)

≤ H(K) – H(K | K)

114
 Chapter 6-Biometrics

= H(K) – H(K | K). (from manipulating the H function)

Hence, for “perfect secrecy”, we have the conditions

H(M) = H(M | C) = H(K | C) ≤ H(K) – H(K | K),

where the maximum source entropy and the key entropy, given the cipher
text, are reduced with the amount H(K | K). This is the price for having a
noisy key.

6.1 Biometric Authentication

Biometric identifiers are the distinctive, measurable characteristics used to
label and describe individuals. The biometric authentication scheme as
developed by Juels-Wattenberg (JW) [41] can be seen as a noisy cipher
system, see Section 6.3. The biometric bn acts as a binary key of length n at
encryption and as a noisy binary key at decryption. The message mk of k bits
is encoded as code word cn of length n using an error correcting code. The
result (bn  cn) is stored in a data base (enrollment), where  is modulo-2
addition. At authentication, the code word is retrieved from the data base and
added to the “noisy” biometric. The decoder gives the correct mk in case the
number of errors is within the error correcting capability of the code. We
will analyze the performance later.

en noise

bio bn bio bn

data
cn + bn base
cn cn + en
encoder decoder

mk mk

Figure 6.3 Schematic representation of the JW scheme

115
 Chapter 6-Biometrics

The JW biometric authentication scheme can also be seen as a wiretap

channel, see Figure 6.4. Encoding is done as in Figure 6.3. The legal
“receiver” has the observations zn = cn  bn and bn = bn  en, where en is an
error vector (the difference between bn and b n). This legal receiver can
calculate yn = cn  bn  b n = cn  en. A data base observer has the
availability of zn = cn  bn. Therefore, Figure 6.3 can be redrawn as the
wiretap model of Figure 6.4. For binary inputs and binary symmetric
channels, the maximum “secrecy capacity” is

Cs  max ( I(C n ; Y n )  I(C n ; Z n ))

P (C n )
 n (H(B)  H(E)), (6.1)

which is the difference in biometric - and noise entropy.

The observation that the legal user has the availability of the output of two
parallel channels, see Figure 6.5, can also be applied to the wiretap channel
to improve the secrecy capacity, see Chapter 5. In this case,

Cs   n ( H ( B)  H ( E ))  max {H ( Z n | Y n )  H ( Z n | C n Y n )} (6.2)
P (C n )

which enlarges the secrecy capacity as given in (6.1). The problem that
remains is the quantification of (6.2).

en
k n
m c legal
encoder yn = cn+en

illegal
zn = cn+bn

bn

Figure 6.4 Wiretap representation for the JW scheme

116
 Chapter 6-Biometrics

6.2 Biometric Reconstruction

The first problem we consider is that of reconstructing original biometric
data given a noisy version of the biometric data and some related data
previously stored in a data base. The condition is that it is difficult or almost
impossible to guess the original biometric data from the stored data
[44,45,46,49].

Enrollment
The reconstruction scheme uses the parity check matrix HT for a length n
linear block code with k information symbols [42]. The parity check matrix
HT has n rows and (n-k) columns. Hence, the inner product of a biometric
vector bn with the parity check matrix gives as a result a “syndrome vector”
sn-k = bn HT of reduced length (n-k), which a server stores in the data base.

en
k n
m c
encoder yn = cn+en legal

zn = cn+bn
wiretapper
bn

Figure 6.5 Wiretap representation for the improved JW scheme

Reconstruction
We assume that, at the legal reconstruction phase, the noisy vector b n is
offered to the server, where bn = bn  en and en is an error vector changing bn
at positions where the error vector en has ones. To reconstruct the original
biometric vector bn, the server calculates bn HT  sn-k = bn HT  bnHT =
enHT. A regular decoding algorithm for the corresponding error correcting
code subsequently produces en and thus reconstructs bn = bn  en. Of course,
knowledge of the statistical properties of en is very important for the
decoding algorithm.

117
 Chapter 6-Biometrics

The correct vector bn can be used for en- or decryption in a key-based

entrance system. Incorrect decoding, caused by a b n with un-correctable
errors, leads to the impossibility to the further use of bn.

We use as an example an RS code over GF(q = 2m) of length n = 2m - 1. The

redundancy is n-k symbols, and p is the symbol error probability. The
important performance parameters to consider are the False Rejection Rate
(FRR) and the False Acceptance Rate (FAR).

bn = bn  en bn

authentication reconstruction

HT

enrolment
bn enHT en
HT decoder

Figure 6.6 Biometric reconstruction scheme

False Rejection Rate

The FRR is defined as the probability that a legal user is not accepted by the
system. It can be upper bounded by the probability that more than (n-k)/2
errors occur, i.e.

n
n i
FRR    p (1  p) n i  (np)(n - k 1)/2 . (6.3)
i
 n  k 1  
i 
 2 

False Acceptance Rate

Suppose that a decoding algorithm only accepts syndromes resulting from
correctable error patterns and an illegal person is assumed to produce a
random syndrome. Then, the FAR is the probability that a random syndrome
is accepted as valid, i.e.

118
 Chapter 6-Biometrics

(n - k)/2   n 
FAR    (n  1)  (n  k)  q  (n  k)/2 .
i
(6.4)
i 0

From (6.3) and (6.4), we see that the FRR and FAR can be reduced by
increasing (n-k). Calculation of the FAR and the FRR are the same as for a
noisy communication channel with the same parameters.

Correct guessing probability

Another measure of performance is the probability that an illegal user of the
system guesses the correct biometric with or without the stored syndrome
from the data base. Assume that an illegal user always guesses the biometric
bn  Bn, where Bn is the set of possible biometrics, with the highest
probability of occurrence, thus minimizing the average probability of
guessing error (Maximum Aposteriori Probability, MAP). Using this
principle, without data base knowledge, the correct guess probability

Pguess (correct)  max P(b n ) . (6.5)

bn  Bn

An illegal user, with data base knowledge of sn-k, can improve this
probability by guessing the bn for which P(bn stored as sn-k| sn-k) is maximum.
Let Sn-k be the set of possible syndromes, then

Pguess (correct | s n - k )  max P(b n | s n - k ) .

n n n -k
B :b  s

The average probability of correct guessing is

P guess (correct | s n - k )   P(sn - k ) max P(b n | s n - k )

s n - k S n - k B n : b n s n - k

  max P(b n )P(sn - k | b n ) (6.6)

s n - k S n - k B n : b n s n - k

 q n - k max P(b n ).
Bn

119
 Chapter 6-Biometrics

The upper bound on the correct guessing probability in equation (6.6) is a

factor qn-k higher than (6.5).

For every particular syndrome, there are only qk candidate biometric words
and thus, the average probability that we have a correct biometric bn can be
bounded as

q k  Pguess(correct | s n -k )  qn k max P(bn ).

Bn
(6.7)
For a small value of k, a high correct guessing probability for the illegal user
can be expected, whereas a large value of k makes it more difficult to guess
the correct biometric.

Using the concept of entropy with base q, the entropy H(Bn | Sn-k)  k
symbols, because there are only qk candidate biometrics for a particular
syndrome vector sn-k. The entropy H(Sn-k) ≤ (n-k), since there are qn-k
possible syndromes. Thus, since H(Sn-k) + H(Bn | Sn-k) = H(Bn) + H(Sn-k | Bn)
= H(Bn), we obtain

k  H(Bn | Sn-k) ,

 H(Bn) – (n - k). ( 6.8)

We call (n-k) the entropy loss or leakage rate, see also (6.6), where we lose a
factor (2m)n-k in the guessing probability. In information theory, the concept
of typicality can be used to show equivalence between (6.6) and (6.8).

6.3 The Juels-Wattenberg Scheme

There are several ways to describe biometrics for authentication purposes.
Biometric data can be given as a collection of different symbols in an
ordered or un-ordered way. In particular cases, biometric data is given by the
possession of particular important properties. At authentication these
properties are checked on their presence. The properties might change,
disappear or other properties appear as important. When the biometrics are
used as a password, one has to deal with these errors without sacrificing the
security. JW designed a scheme where both aspects, are incorporated.

120
 Chapter 6-Biometrics

6.3.1 Juels-Wattenberg Scheme with a biometric of length n

The original JW scheme using general linear codes is described for

biometrics bn of length n.

?
Pk hash(Pk) = hash(Pk) Pk

encoder decoder

cn
corrupted code word cn

bn bn
Figure 6.7 JW scheme for linear codes

Enrollment
We add a biometric vector bn of length n with components from GF(2m) to a
randomly chosen code word cn from a scheme specific RS code. The code
word is generated by a random vector Pk. We store the sum cn = bn  cn in a
data base. In addition, we also store the hash(Pk) in the data base.

Authentication
At authentication we add again the relevant biometric and the result is cn 
bn = bn  cn  bn = en  cn, see Figure 6.7. From this, we can retrieve the
original code word cn and thus Pk if the number of errors is within the
decoding region. The hash(Pk) can be used to check for correct decoding.

False Rejection Rate

The FRR is upper bounded by the probability that we have more than dmin/2
errors in the biometric bn. For RS codes over GF(2m) with minimum distance
dmin= n-k+1,

121
 Chapter 6-Biometrics

 n -k 
n
n i  2  1
FRR    p (1  p)
i
n  i  (np)  
,
 n k 
i  1
 2 
.
which is the same as (6.3)

False Acceptance Rate

For the FAR, we calculate the probability that a random vector bn is such
that bn  bn = en is within the decoding radius of the code word cn, i.e.

 n -k 
 2  n k
  n  
FAR    (n  1)  n  q  k  q
i
2 .
i 0  

This FAR is a factor q-k better than (6.4). The calculation is based on a
random biometric b, whereas it can be expected that the vector bn probably is
not a random vector, but drawn from the same set of biometrics as the
original one.

Correct guessing probability

The correct guessing probability is the same as given in Section 6.2.

6.3.2 Juels-Wattenberg scheme with a biometric of fixed length t < n

We describe a particular implementation that uses RS codes as basic

component. Let the RS code be over GF(q = 2m), i.e. length n = 2m - 1. The
biometric is characterized by t different values less than 2m and is specified
by bt = (b1, b2, ···, bt), bi  GF(2m), bi  bj, The scheme works as follows.

Enrollment
1. generate a random word Pk of k symbols from GF(2m), and store the
corresponding hash value, hash(Pk), in the data base. The hash(Pk)
produces a value from which we cannot reconstruct Pk;
2. encode Pk with an RS encoding matrix as cn;
3. the t different values from bt are used as indicators for the positions in cn
where the code word symbols are fixed. In all other (n-t) positions, the

122
 Chapter 6-Biometrics

symbols are changed in a random way from their original value. A code
word cn is created with n - t errors;
4. the word cn is stored in the data base.

Authentication
We assume that the biometric has r error symbols. Hence, the biometric
points at t – r correct positions and r incorrect positions. For the decoding the
remaining positions are considered to be erasures. Since the code has
dimensions (n,k), minimum distance n-k+1, the correct code word cn and
thus Pk can be decoded for

(n – k + 1) – (n - t)  2r +1.

For (t – k)  2r no decoding error can occur. After decoding Pk, the hash(Pk)
can be calculated and compared with the stored value from the data base at
enrollment.

Note We can also let bt determine t incorrect positions. Correct decoding is

then guaranteed for

(n – k + 1) – t  2r + 1,

or (n – t - k)  2r. A schematic representation is given in Figure 6.8.

?
k
P hash(P ) = hash(Pk)
k
Pk

encoder decoder

error corrupted code word position

generator selector

position bt
selector

bt

Figure 6.8 Representation of the JW scheme using RS codes

123
 Chapter 6-Biometrics

We use this description of the JW scheme, because it fits the use of RS codes
very well and it relates the JW scheme with the Juels-Sudan (JS) hereafter.

6.4 The Juels-Sudan (JS) Scheme

The JS scheme [47] is a biometric authentication scheme also based on RS
codes. We describe a particular version of the JS scheme.

Code words of an RS code over GF(q = 2m) are generated by evaluating a

random information polynomial P(X) of degree k-1 (equivalent to a vector
Pk of length k with elements from GF(2m)), for X = i, i = 0,1, …, n-1;  a
primitive element of GF(2m); n = 2m -1. The k symbols from P(X) can be
hashed, as for the JW scheme [41] and stored as hash(P(X)) in the data base
for further use.

Enrollment
We assume that bt = (b1, b2, ···, bt), bi  GF(2m), bi  bj, represents the
biometric at enrollment. Then:

1 choose random P(X) of degree k-1;

2 store : cn = (c0, c1, ··· , cn-1 ), where
ci = P(i) for i  bt,
i
ci  P( ) in (n – t) other positions.

As mentioned before, after enrollment the word bt can change due to errors
like measurement imprecision, aging, etc. We assume that for a particular
user at authentication we offer a word bt from t biometric properties. The
procedure at authentication is described as follows.

Authentication
Given bt = (b1, b2 , ···, b t), bi  GF(2m), bi  bj .

1 evaluate P(i ), for i  bt;

2 declare the remaining positions as erasure;
3 decode P X);
4 give Reject when hash(P(X)) unequal hash(P(X)), otherwise Accept.

124
 Chapter 6-Biometrics

Note, that bt determines t positions in cn, whereas the values at these

positions are uniquely determined by the polynomial P(X). The values at the
(n - t) other positions differ from the evaluation of P(X).

False Rejection Rate

The legal user, having a biometric bt, considers t positions, of which possibly
a maximum of r positions are wrong. The remaining (n - t) positions are
considered as erasures. Correct decoding (the hash values comparison gives
Accept) is guaranteed if (n–t+2r+1)  (n–k+1) or 2r  (t–k). The
probability that there are more than (t-k)/2 symbol errors in t positions, for a
symbol error rate p, thus gives an approximation of the FRR

FRR  (tp)1+(t-k)/2..

Making k larger will reduce the error correcting capability of the decoder
and thus the FRR will increase.

False Acceptance Rate

For the FAR we assume that an illegal person produces a random vector of t
positions. The FAR is then given by the probability that a random vector at
authentication has at least k correct positions from cn that lead to the
reconstruction of P(X). Hence, under this condition,

2
 t  n  k   n   t   n   t   t   k
FAR     /      /        q . (6.9)
 k  t  k   t   k   k   k   k 

For t = k, the FAR is roughly equal to q-k, the same as for a direct guess
without using the biometric.

Correct guessing probability

Since the data is stored in a public data base, an illegal user of the data base,
trying to decode on the basis of the observed data, has n-t errors in n
symbols and thus correct decoding is “not” possible for 2(n-t)+1 > n–k+1
or 2t < n+k. One possibility is to guess k correct positions out of n and try
to decode P X with (n-k) erasures. Note that t positions out of n are correct.
Hence, the probability of success

 t  n  t 
P(successful guess) =   /    q  k . (6.10)
k k k

125
 Chapter 6-Biometrics

Without biometric, the probability of a correct guess is q-k. Hence, we have a

loss due to the fact that the biometric bt has t properties. An illegal user
could also guess P(X) given cn using the Maximum Aposteriori Probability
(MAP) principle and thus:

P guess (correct | c n )   max Pr(P k | c n )Pr(c n )

P
C
  max Pr(P k )Pr(c n | P k )
P
C
 q  k (q  1)  (n  t)  max Pr(b t )
C b t :(P k , c n )  bt
n
 q  t k
   q max Pr(b t ).
 q - 1  bt

As in (6.7) we have

n
 k n  q  t k
q  P guess (correct | c )    q max Pr(b t ). (6.11)
 q - 1  bt

Comparing (6.11) with (6.7) we see an exponent (t-k) instead of (n-k).

6.5 An Improved Version of the Juels-Sudan Scheme

Dodis et al. [43] published an improved version of JS. In this version only t
instead of n symbols are stored in the data base. Performance can be shown
to be the same as for the original scheme, when n symbols are stored. It can
be described as follows:

Enrollment:
Given the biometric bt =(b1,b2,···,bt), bi GF(2m), bi  bj.
1. choose a random secret P(X) of degree k-1;
2. calculate Q(X) = P(X) + (X-b1)(X-b2) ···(X- bt);
3. store Q(X) of degree (t-1).

126
 Chapter 6-Biometrics

Authentication:
Given bt = (b1, b2 , ···, b t), bi  GF(2m), bi  bj .
1. evaluate Q(bi ), i = 1,t. Q(bi ) = P(bi ) for bi = bi;
2. decode P(X).

For bt, with at least k correct values, we have correct decoding of P(X). If
we look at the vector Qt that corresponds to Q(X), we conclude that Qt plays
exactly the same role as the vector cn in the JS scheme. One of the problems
with the JS scheme is the choice of biometrics in the application. A
fingerprint-based fuzzy vault implementation can be found in Nandakumar
et al. [48].

Note The JS scheme is close to the JW scheme when we consider RS codes.

In the JW scheme, we store cn + bn in the data base, where cn is generated by
Pk. At authentication, we add bn to the stored result and thus we obtain cn +
en. If we assume not more that t errors, we can decode the original code
word. The difference with the JS scheme is that bn can change the whole
code word cn, whereas in the JS scheme only a particular set of values is
changed. If bn contains exactly t positions, the two schemes are exactly the
same.

noiseless Shannon encryption

noisy Shannon encryption

biometrics as a key

description as a wiretap channel

coding: probabilistic biometrics

coding: fixed set of biometric

Figure 6.9 Schematic form of the content of Chapter 6

127
 Chapter 6-Biometrics

6.6 Concluding Remarks

This chapter gives an interesting application for the wiretap channel. We
start with a noisy crypto scheme as defined by Shannon and relate this
scheme with the noisy fingerprint authentication as developed by Juels and
Wattenberg. The scheme uses a probabilistic description of the biometric.
Therefore, we can describe this biometric authentication principle using a
wiretap channel model. In [79], an algorithm is described that estimates the
entropy of finite length biometric data.

We conclude with an authentication scheme derived by Juels and Sudan.

This scheme is based on a maximum number of properties from the
biometric. It is interesting to see the connection between the Juels-Sudan
scheme and the Juels-Wattenberg scheme. We derive and compare the
performance parameters False Acceptance Rate, False Rejection Rate and
probability of a successful attack. In schematic form, the content of Chapter
6 is given in Figure 6.9.

128
 Chapter 7-Codes with Constraints

Chapter 7

RS Codes with Constraints

This chapter contains four topics that can be seen as coding with restrictions,
also called constrained coding. The first topic is the avoidance of certain
symbols to occur in the output of an RS encoder. The second topic is the
combination of RS and Run-length constrained coding with application in
bandwidth limited channels. The third topic introduces the distance profile of
a code and we conclude with RS codes for which the maximum number of
symbols of the same type is bounded.

7.1 Symbol Avoidance

The first problem is to generate RS code words over GF(q) in such a way,
that particular symbols do not occur in the code word. To achieve this, we
use a method described in a paper by G. Solomon [57]. The method is
described for the avoidance of one symbol, but can be extended to exclude a
particular set A of symbols with cardinality |A|.

Iκ 0 P
Gk,n = k
0 Ir Q

Figure 7.1 Parameters for a systematic RS-encoding matrix

129
 Chapter 7-Codes with Constraints

For instance, if we want to use an RS code over GF(11) with code words of
length 10 and code word symbols from {0, 1, ···, 9}, we have to exclude the
symbol 10 in the encoder output.

In Figure 7.1 we give the parameters of the RS code in systematic form that
are of importance for the description. Note that, κ + r = k. We encode a
message as

(mκ,sr) Gk,n = cn,

where mκ is the information to be encoded and sr a control word of length r.

The κ q-ary information symbols are pre-coded such that they do not contain
a particular set A of symbols. The r control bits are used to manipulate the
check part (last n-k symbols of a code word) such that symbols from A do
not occur in the check part. From the constraint it follows, that the maximum
number of control words we can choose is (q - |A|)r.

κ RS (mκ,0r)Gk,n
information check part
symbols ϵ Ac generation

n-k (mκ,sr)Gk,n
check symbols

construct (0κ,sr)Gk,n
control
word sr

Figure 7.2 Concept of the control word generation

Property A control word sr can be used to create any symbol in a particular

position in the check part.

The property follows from the fact that, the minimum distance for this
systematic RS code is dmin = (n - k + 1), and thus the last (n - k) symbols of
any row are non-zero. Furthermore, since every element in the Galois Field
has an inverse, we can create any symbol in a particular position in the check
part by using a particular control word sr.

130
 Chapter 7-Codes with Constraints

Encoding
Let the first κ pre-coded information symbols (no symbol from A) generate a
check part. A particular set of control words has to be selected that generate
check parts such that the symbols from A do not occur in the summation of
the check parts generated by the information and control input, see also
Figure 7.2. Every symbol in the check part generated by the information,
eliminates a maximum of |A|(q-|A|)r-1 control words that cannot be used. If
the total number of eliminated control words is less than (q - |A|)r, we can
always find a particular choice for the control input that realizes a check part
without the undesired symbols. Thus, for

|A|(n-(κ +r)) (q-|A|) r-1 < (q-|A|)r

or

 q | A | 
n  (κ  r)    , (7.1)
 |A| 

this condition is fulfilled.

Example For |A| = 1, r = 1, q = n+1 (RS), the condition is always fulfilled.

Example For |A| = 2, r = 1, the redundancy n-(κ+1) < (n-1)/2, or κ > (n-
1)/2.

Example Take an (n = 7, k = κ + r = 3) RS code over GF(23). The field

GF(23) is defined by a primitive polynomial, P(X) = X3 + X + 1.
The generator matrix

100 6167
G3,7 = 010 4155 .
001 3123

By choosing r = 1, we have κ = 2. Let A = {7}. From G3,7, we have the

following control words we can use when we encounter an information
check part containing the symbol 7,

[ 0 0 0 0 0 0 0], [ 0 0 1 3 1 2 3], [ 0 0 2 6 2 4 6],

[ 0 0 3 5 3 6 5], [ 0 0 4 7 4 3 7], [ 0 0 5 4 5 1 4],
[ 0 0 6 1 6 7 1].

131
 Chapter 7-Codes with Constraints

The word

[007 2752]

cannot be used to eliminate the symbol 7 since it has 7 in its systematic part.
To illustrate the elimination of symbol 7, we pick one code word with the
symbol 7 (in the check part) from the list of code words produced by G3,7
using the two information digits (0,3). The specific code word is

c7 = [0 3 0 7 3 4 4].

Taking a suitable control word as [0 0 3 5 3 6 5] from G3,7, we obtain the

final encoded word without the symbol 7 as [ 0 3 3 5 6 3 2 ]. It can also be
verified that either [0 0 5 4 5 1 4] or [0 0 6 1 6 7 1] is a suitable control word

7.2 Constrained Coded Modulation

Run Length Limited (RLL) codes are used in communication channels with
power spectral limitations. The limitations (constraints) are translated into a
longest (k-constraint) and a shortest run of transmitted symbols that are the
same (d-constraint). The maximum run guarantees a variability in the
transmitted symbols that improves the synchronization. In our case we
assume k = ∞. The shortest run length constraint limits the power spectral
density in the higher frequency part of the power spectrum. For further
information, we refer to the textbook by Schouhamer-Immink [32].

We describe the application of constrained coded binary modulation in the

time domain. We combine error correcting codes and RLL codes before
binary modulation. The RLL code is designed in such a way, that the
minimum spacing between two transitions in the binary RLL word is at least
the same as for the un-coded symbol duration. In this way, we obtain a
system where we have almost no change in the Power Spectral Density
function but a net positive coding gain as for coded modulation invented by
Ungerböck [58].

7.2.1 System design

In CD type of recording systems one often combines an error correcting

code RS code with an RLL constrained code [55]. The advantage is that

132
 Chapter 7-Codes with Constraints

errors in the constrained code can be seen as symbol errors for the error
correcting code RS code.

Example An example of a combination is given in Figure 7.3, where we

encode 223 x 8 bits into RS code words of length 255 symbols of 8 bits and
convert each RS symbol into a constrained RLL code word of length 14.
The RLL code words are constructed in such a way that they can be
concatenated without violating the d-constraint. For this, one can use stuff
bits (3 bits for the CD length-14 RLL code) or the principle of RLL 1-
symbol look-ahead encoding, see Hollmann [59]. The resulting bits are
binary modulated, where the carrier period is a multiple of the RLL symbol
duration.

The RS code needs an expansion of the time needed to transmit the

information, whereas the constrained RLL code reduces the transmission
time. The goal is not to change the total time to transmit the k information
symbols. Since we aim at short code word lengths, we can apply soft-
decision decoding on the constrained RLL code words if in addition to the d-
constraint, we also have a minimum distance between any two RLL code
words larger than 1.

error conversion modulator

correction to RLL signal
code generator
Information
RS RLL
223 x 8 255 x 8 255 x (14+3)

Figure 7.3 Principle encoding/modulation scheme

7.2.2 Idea for the modulation

We use binary modulation to transmit the constrained words. The RLL

constraint d and the RLL symbol duration ’ are chosen such that ’(d+1) =
. As a consequence, the RLL symbol is repeated at least d+1 times to give
a new symbol of minimum duration . The minimum separation between
two transitions in the binary RLL word is at least . Therefore, the Power
Spectral Density (PSD) can be expected to stay roughly the same as for the
original un-coded binary word. In addition, the modulator output also has a

133
 Chapter 7-Codes with Constraints

constant envelope output. In Figure 7.4 we give the Power Spectral Density
of the modulated RLL words as a function of the parameter d using Binary
Phase Shift Keying (BPSK).

Figure 7.4 Normalized Power Spectral density for different values

of d for BPSK signaling

7.2.3 Idea for the RLL-encoding

We use Figure 7.5 to explain the main idea in more detail.

1. we first convert binary words of length k information bits into code

words of length n from an error correcting code and symbol duration .
The efficiency of encoding is k/n;
2. the n symbols are converted into constrained binary words of length m
with symbol duration ’ and a minimum run of d+1 symbols that are the
same. The efficiency of the conversion is RRLL = n/m.

Now, consider a time frame of length T with n = T/ code word symbols. In
the same time T, we can have m = T/’ symbols from the constrained RLL

134
 Chapter 7-Codes with Constraints

code. Hence, in this case we have the relation T = n = m’ and using the
condition that ’(d+1) =  we have RRLL (d+1) = 1. If in addition we use an

Figure 7.5a Message of length 6, encoded with rate 6/8

Figure 7.5b RLL code, d+1=2, with 64 code words of length 12

135
 Chapter 7-Codes with Constraints

error correcting code with efficiency k/n, the final condition of no change in
the PSD or “no bandwidth” expansion is

(k/n) RRLL (d+1) = 1. (7.1)

Figure 7.5 illustrates the idea in the time domain. Six information symbols
are encoded into eight code symbols. The eight code symbols are translated
into twelve RLL symbols. The overall rate satisfies (7.1).

The research problem to solve, is the design of the combination of error

correcting codes and constrained codes such that (7.1) is fulfilled (the d-
constraint). The additional problem is the design of a demodulator. We will
discuss a particular system design in the next section.

Table 7.1

Code Table: RLL(d+1=2), RRLL=3/5

message code word message code word

0 00011 4 00001 or 00110
1 00111 5 11110 or 11001
2 11000 6 01111 or 10011
3 11100 7 10000 or 01100

Example In Table 7.1 we give the RLL words for (d + 1) = 2 that can be
implemented using a 1-symbol look-ahead method. For instance, if we store
message 4 we can choose between 00001 and 00110. The code word is
selected in such a way that the next code word does not violate the d-
constraint. Thus, if the next message is 00011 we use 00110, whereas for
message 2 we use 00001. In this way, we do not have to use extra bits
between the RLL words to satisfy the d-constraint as is in recording systems
like CD, Hollman [59].

136
 Chapter 7-Codes with Constraints

If we use an RS code over GF(23), the binary code symbols are of length
three. Every symbol can be converted into an RLL code word. For k = 5 and
n = 6, dmin = 2 and the value of RRS RRLL (d+1) = 1. After demodulation,
the RLL symbols can pass the likelihoods for every possible symbol to the
RS decoder. Since dmin = 2, we expect a coding gain of 3 dB without any
bandwidth expansion, see Ungerböck [58].

Example The RS code we focus on is of length n = 255 symbols of 8 bits.

The code has minimum distance n – k + 1, where k is the number of
information symbols. For the constrained code, we select an RLL code with
parameter d = 1 or 2. The research problem, solved by Mengi [60,61], is the
construction of RLL codes that satisfy the following constraints:

1. RLL code words can be concatenated without violating the d-constraint;

2. RLL code words must have a minimum distance larger than 1, to
facilitate soft-decision RLL decoding;
3. the number of code words is at least 256;
4. the combination RS, RLL satisfies (7.1).

Ad 1). The first condition can be solved using the approach by Hollmann
[59] for the construction of block decodable RLL codes.

Table 7.2

Example of RLL, RS combinations that

satisfy RRS RRLL (d+1) = 1 and dmin = d+1

d RRLL dmin RRLL (d+1) RRS

1 8/14 2 8/7 223/255

2 8/21 3 8/7 223/255

Ad 2). The Hollmann construction is given for dmin = 1. In [60,61], an

extension can be found for dmin > 1. The fact that the minimum distance is
larger than one, gives the opportunity to do soft-decision on the RLL code
words.

Ad 3). RLL codes are generated that satisfy the d-constraint. Then, we make
a selection of the resulting code words that lead to an increase in minimum
distance. The number of resulting code words should be larger than 255 to
be able to make a concatenation with the 8-bit RS symbols.

137
 Chapter 7-Codes with Constraints

Ad 4). If the product RRLL(d+1) < 1, we can select the corresponding RS

code to satisfy (7.1). In Table 7.2, we give some of the constructed RLL
codes such that (7.1) is satisfied. More constructions can be found in [60,61].

Demodulation The last part of the system is the demodulator. We assume

that the carrier frequencies are multiples of 1/’ instead of 1/ to obtain
phase continuity. At the receiver the demodulator therefore changes the
integrate-and-dump time-interval to ’. For non-coherent FSK we assume
furthermore, that there is a constant phase within an RLL-block. After
demodulation, we have 255 blocks of m soft values. For every block we find
the RLL code word index i that maximizes

m
Si   cijs j , c ij   /  1; i  1,28 , (7.2)
j1

where sj is the soft output of the demodulator per interval ’ and cji the j-th
symbol for block i. The complexity of this action is proportional to the
number of code words times the code word length m. After this operation,
we can apply hard-decision RS decoding.

Performance
The performance depends on the soft-decision decoding of the RLL code
words and the decoding of the RS code. We assume Gaussian noise with
PSD = No/2. Since  = (d+1)’, the RLL soft-decision error rate PRLL is given
by

 2E   2E b 
PRLL  Q d min s   Q d min  . (7.3)
 
N0   (d  1) N 0 
  

For dmin = d+1, the RLL symbol error rate is of the same order as for un-
coded transmission. Therefore, we expect the coding gain to come from the
RS code. In Figure 7.6, we give the resulting “waterfall” curves for the
system. Three RS codes are simulated with dmin = 1, 2, 3 respectively. The
best performance is given by the (255,223) RS code combined with the RLL
d = 2 and dmin = 3 code. The RS code has minimum distance 255 - 223 = 32,
and the decoder can correct up to 16 symbol errors. Different code
constructions and combinations can be found in [60,61].

138
 Chapter 7-Codes with Constraints

Figure 7.6 Performance curves for several RS-RS combinations

Conclusion
We describe a coded modulation scheme where the redundancy for error
correction is obtained from relaxation of the timing. The encoded symbols
have the same minimum spacing between two transitions as for the un-coded
case, but the symbols might be longer in discrete steps depending on the
parameter d. For a particular example, we show a coding gain of about 4 dB.
An additional advantage is that we have a constant envelope modulated
signal. This can be of importance for systems where we have an absolute
power constraint as for instance in the “Power-line Communications
CENELEC band.” Our system is considered for AWGN, but will also work
for other types of noise, since we have a block wise detection and decoding
without error propagation.

139
 Chapter 7-Codes with Constraints

7.3 Optimum Distance Profile

Motivation Error control coding is very often a fixed part of a
communication systems design. To change a code is impossible due to
standards and already implemented decoding algorithms. In cognitive
systems, we want to adapt the efficiency of a transmission scheme to the
actual circumstances. For this we have to be able to change the modulation
and also the error-correcting or error-detecting code.

Specify a code
↓
Start with an encoding matrix
↓
Generate the equivalent matrices
↓
Calculate the distance profiles
↓
Output the Optimum Distance Profile

Figure 7.7 Concept of the optimum distance profile generation

The transport format combination indicators (TFCIs) are widely applied in

CDMA systems, see [55,56]. In the realization of TFCI, input bits are used
to combine some basis code words of a linear block code. When the number
of the input bits increases or decreases, some basis code words will be
included or excluded, respectively. In this process, a general consideration is
how to realize large minimum distances of the generated sub-codes.

Some concepts are well known, such as puncturing or

shortening/lengthening an encoding matrix of a linear code. In this way, the
efficiency of the code is changed by changing the length n of the code
words. We choose another method: row extension/deletion of the encoding
matrix for an (n,k) code, and thus change the parameter k.

Row deletion means that we delete one row from the encoding matrix such
that it generates a new (n,k-1) code. Here, the effect is reduction in the code
efficiency from k/n to (k-1)/n but at the same time a possible improvement in
the minimum distance [71].

140
 Chapter 7-Codes with Constraints

We first consider one-by-one row deletion from a generator matrix Gk,n that
generates the starting “mother code”. The mother code has minimum
distance dk. We delete a row from this matrix and obtain Gk-1,n with
minimum distance dk-1. From Gk-1,n we can again delete a row and obtain
Gk-2,n with minimum distance dk-2. The process ends when we obtain a single
row from G1,n. This process generates a distance profile

DPBdel(G) = (dk, dk-1, ···, d1).

A distance profile DPBdel(G) is optimum ( ODPBdel ) if

~
di = di , for k  i  t+1, t  1,
~
dt > d t ,
~
for any equivalent encoding matrix G k , n generating the same “mother
code”.

Example The (7,4) Hamming code can be generated by the encoding matrix

1000111

0100110
G4,7 = .
0010101

0001011

The minimum distance is three. Deleting row-by-row, starting from the

bottom, we get the distance profile

DPBdel(G) = (3, 3, 3, 4). (7.4)

~
The same code can be generated by an equivalent matrix G , i.e.

141
 Chapter 7-Codes with Constraints

1011100

1110010
~
G 4,7 = .
0111001

1111111

Deleting the last row at first, gives the distance profile

~
DPBdel( G )= (3, 4, 4, 4), (7.5)

which is better than (7.4) and the best possible for this code. For any linear
code, we can find the Optimum Distance Profile, ODPBdel, by taking the best
possible distance profile over all possible equivalent encoder matrices, see
Figure 7.7. Of course, this is a complicated task. In [54] Johannesson defines
the Optimum Distance Profile for convolutional codes.

Row extension means that we add a row to the encoding matrix such that it
generates a new (n,k+1) code. The effect is an improvement in the efficiency
from k/n to (k+1)/n, but at the same time a possible degradation in the
minimum distance.

We start with a single particular row from a code generator matrix and
extend the matrix row-by-row. Every matrix extension must generate code
words from the desired “mother code”. In every step, we select a row such
that we maximize the minimum distance of the constructed code generated
by code generator matrix. The starting code has minimum distance δ1,
determined by the Hamming weight of the first selected row. We add a row
to this matrix and obtain G2,n with minimum distance δ2. We can add a row
to G2,n and obtain G3,n with minimum distance δ3. The process ends when we
obtain the final code generator matrix Gk,n. We generate a distance profile

DPBext (G) = (δ1, δ2, ···, δk).

~
A distance profile DPBext(G) is better than DPBext( G ) when
~
δ i = i for 1 ≤ i ≤ t-1, t ≤ k,

142
 Chapter 7-Codes with Constraints

~
δ t > t .

Example The (7,4) Hamming code can be generated by the encoding matrix

1111111

1011100
G4,7 = .
1110010

0111001

The matrix G4,7 generates a distance profile

DPBext(G) = (7, 3, 3, 3).

Again, this is the best possible profile for the Hamming code. Remark that dk
= dmin, whereas δ1 equals the maximum Hamming weight of a code word.

Example For RS codes, we take as a starting generator matrix

1 1 1  1
1  2  n-1
Gk,n = 1 2 4  2(n-1) .

1 k-1 2(k-1)  (n-1)(k-1)

Deleting row by row gives as a distance profile

DPBdel(GRS) = (n-k+1,n-k+2, , n).

If we start with the first row and extend the matrix row by row, we get

DPBext(GRS) = (n, n-1, , n-k+1).

Note that both these profiles are optimum.

Remark There is an extensive report on the Optimum Distance Profile for

Block codes in [51,52,53,71].

143
 Chapter 7-Codes with Constraints

7.4 Same-weight Code Construction

We give a particular code construction using an (n,k) RS code, such that the
generated code words have the property that all code words have a
maximum of k symbols that are the same.

Motivation In MFSK modulation, symbols from a code word are

transmitted as one of the M possible frequencies. The transmission can be
disturbed by permanent narrowband noise in such a way that the
demodulator outputs a symbol that is always present [62]. An (n,k) RS
encoding matrix containing the all-ones row produces code words that have
n symbols that are the same. Hence, correct decoding will not be possible.
We describe an encoding method that avoids this problem. An extensive
description is given in [63].

Let an RS code be generated by the standard RS encoding matrix given by:

1 1 1 … 1
1  2 … n-1
Gk+1,n = 1 2 4 … 2(n-1) C1
•••
1 k-1 2(k-1) ... (k-1)(n-1) .
1 k 2k ... k(n-1)

The matrix Gk+1,n has the following properties:

- the minimum distance for this code is dmin = n-k;

- since the minimum distance is equal to the minimum number of
differences between two code words, the code words agree in at
most k positions;
- multiples of the first row give n symbols that are the same.

Given the above matrix, we construct a sub-code with the property that all
code words of the sub-code have a maximum of k symbols that are the same.

Construction: Generate code words with the code matrix C1. Add the last
row from Gk+1,n to every code word. After decoding, subtract the last row
from the decoded code word.

As a consequence of the construction, code words with n symbols that are

the same do not occur. Since we always add the last row of G, the all-0 code
word will not occur. We therefore have the following properties:

144
 Chapter 7-Codes with Constraints

Properties
- the “same-weight” code generated by C1 has dmin = (n-k+1);
- all code words have a maximum of k symbols that are the same;
- code words contain at least n/k different symbols;
- the code generated by C1 has efficiency k/n.

Note Since any linear code can be brought in systematic form, the number of
the same symbols for non-zero can be larger or equal to k. Using the
constructed sub-code, we obtain k as an upper bound. Hence, RS codes are
optimum.

We briefly summarize the operations at the receiver side that utilize the code
properties. For simplicity, we suppose that the MFSK demodulator outputs
the presence of a particular frequency. A narrowband disturbance may cause
a particular frequency to be present during a long period of time. We can
detect and delete this continuous presence. Using the same-symbol property,
the positions where the code word symbols agree with the disturbance are
considered as erasure positions. Since there is a maximum of k erasures per
narrowband disturbance, the number of narrowband disturbances that can be
corrected for this idealized situation is NB < (n-k+1)/k. The same situation
occurs when the disturbance blocks the transmission of certain frequencies.
The maximum number of “deletions” for our code is then equal to k and the
same analysis can be made. Performance gets worse when in addition to
narrowband noise we also have background noise.

7.5 Concluding Remarks

This chapter contains several applications of RS codes. We start with the
principle of symbol avoidance, where the output of an RS encoder does not
contain a particular set of output symbols. The idea is from Solomon
Golomb and was published in 1974. Golomb only considered one forbidden
symbol. The principle is easy to extend to more than one forbidden symbol.

The second topic is coded modulation in the time domain. Ungerböck

developed code modulation in the 80s for two-dimensional signal spaces, by
expanding the number of symbols that can be used. In our coded modulation
scheme for binary modulation, we only extend the length of symbols, while
keeping the minimum time between two transitions constant. As a
consequence, the bandwidth needed to transmit a message is not enlarged.
We combine the modulation with look-ahead RLL coding as developed by

145
 Chapter 7-Codes with Constraints

Hollmann, and show that the combination RLL-RS gives a coding gain of
about 4 dB.

The third topic is concerned with a new property of Block codes, the
“Optimum Distance Profile”. This problem can solve the difficulty of
extending/reducing the dimensions of an encoding matrix. We show that RS
codes have the best possible profile.

symbol avoidance

coded modulation

optimum distance profile

same weight construction

Figure 7.8 Content of Chapter 7

The last topic describes a special application of RS encoding such that every
code word contains a desired maximum number of the same type of symbol.
This way of encoding can be beneficial when a channel is disturbed by
narrowband noise.

The idea of constrained coding can be extended to convolutional codes, see

[73], where we make sequences DC-free by using the linearity property of
convolutional codes.

The content of the chapter is summarized in Figure 7.8

146
 Chapter 8-Memories with Defects

Chapter 8

Memories with Defects

8.1 Defect models
Improvements in process technology and clever circuit design make it
possible to produce large memory systems on a chip with a high packing
density. A high packing density has its limits and may cause errors in the
memory cell structure. A popular error model is that of defects. A defect
always produces the same output when being read, irrespective of the input.
This might be “defect-0” or “defect-1”. The model for defective memory
cells is given in Figure 8.1.

0 0 0 0 0 0

1 1 1 1 1 1

no defect defect-0 defect-1

Figure 8.1 Defect model for defect-0 and defect-1

In principle, we can write in the memory in coded or un-coded format. The

reader may use a decoder to give an estimate for the encoded and stored
information. We can distinguish between four situations, whether the
writer/reader knows the value and position of the defects or not. This

147
 Chapter 8-Memories with Defects

information may follow from inspection or by some additional side

information.

The four situations give different results for the maximum storage efficiency
or storage capacity. Assume that the fraction of defect-1 cells is equal to the
fraction of defect-0 cells is equal to p/2. The situations can be described as:

1-p
0 0
p/2 p/2

0 defect known before

WRITE READ
1 defect known before

p/2 p/2
1 1
1-p

Figure 8.2 Erasure channel model for known defects at receiver

1. writer and reader know. In this case, we can avoid the defective cells and
thus the storage capacity is (1-p) bits per cell;
2. writer does not know, but the reader knows. The reader, knowing the
position of a defect can treat this position as an erasure. Hence, the
channel that he creates is an erasure channel with maximum storage
capacity (1-p) bits/cell. The writer should use codes for the erasure
channel to store his information, see the model in Figure 8.2;
3. the reader and writer do not know the defects. The defect can be
considered as a random error and thus, on the average p/2 errors occur
when reading. The channel that results is a Binary Symmetric Channel
with storage capacity 1 – h(p/2), where h(*) is the binary entropy
function, see the model in Figure 8.3;
4. the writer knows and the reader does not know the value and the position
of the defect. This situation is the most interesting one, since we can
design a special type of coding. Kuznetsov and Tsybakov showed that in
this case also the capacity is (1-p) bits/memory cell. In addition to that,
they designed the “additive” coding method that asymptotically achieves
this capacity, see Section 8.2.4.

148
 Chapter 8-Memories with Defects

We continue with situation 4 and we assume, that the writer is able to

determine the location and value of a defect before using the memory.

1 – p/2
0 0
p/2
WRITE READ
p/2
1 1
1 – p/2

Figure 8.3 BSC model for unknown defects at writer and reader

One of the possibilities is to use an error correcting code to store words in

the memory. For a single defect per word, one could think of using the
Hamming code. For instance, the (7,4) Hamming code can correct one error
in 7 bits and stores 4 bits of information. Kutznetsov and Tsybakov initially
worked on the (64,58) extended Hamming code that can correct one error or
detect two errors (minimum distance 4), at the expense of 6 bits redundancy.
Their task was to reduce the redundancy and their solution will be explained
in the sequel. The idea [64,66] finds wide spread application, for instance in
coding for Write Once Memories or Flash memories.

8.2 Defect Matching Codes

8.2.1 One defect in n positions

Suppose that words of length n have a maximum of one defect per word. By
inspection, the writer knows the value of the defect and encodes a word xn-1
as (0,xn-1) or (1,xn-1), respectively, where xn-1 is the complementary word for
xn-1. One of both words can always be stored error free and thus the
redundancy is only one bit, irrespective of the value n. The reader can
always find back the original message by inspecting the first bit of a stored
word. This example shows that knowledge about the defect helps to reduce
the redundancy needed for defect error correcting.

Example Let n = 7 and a defect-0 at position 3. The information to be stored

is xn-1 = ( 0 1 1 0 0 0 ). Since (1,xn-1) matches the defect-0 in the third

149
 Chapter 8-Memories with Defects

position, we store ( 1 1 0 0 1 1 1). The information can be retrieved by

inverting the last six bits.

8.2.2 n-1 defects in n positions

Another extreme example is the one that occurs when in n bits, n-1 bits are
defective and only one bit is error free. In this case, since we can observe the
value of the defects, we can always make the parity of the word even or odd
with the remaining error free position. Hence, we store one bit of
information in n positions.

Example Suppose that for the vector (1 0 1 1 0 - 1 1) only position six is

free to chose. Then, a value 0 at that position makes the work parity equal to
1, whereas a value 1 makes the parity equal to 0.

Note Both strategies are optimal. In the first case, n-1 bits can be stored in n
positions and the data storage efficiency R = 1 - 1/n. In the second situation,
we can store only one bit in n positions and the storage efficiency R = 1 -
(n-1)/n = 1/n.

8.2.3 Two-defects in n positions

We proceed by designing a method to handle a maximum of two defects in a

word. A code word cn is called defect compatible or matching the defects, if
it can be stored without any changes, i.e. the components of a code word
agree with the values of the defects. The code word itself depends on the
defects and the message xk to be stored.

We first construct a code matrix for which any pair of bits is present in some
row. If this matrix exists and every row has a unique prefix, then this matrix
can be used to match defects. The construction is as follows:

1. take all binary vectors of length (2α - 1) and weight α as columns of the
code matrix;
2. add the all zero row to the matrix;
3. select log2α columns such that all rows of the 2α × log2α sub
matrix are different.

Ad 1) if we compare two arbitrary columns, then they contain at least the

combinations (0,1), (1,0) and (1,1) in one of the rows. This follows from the

150
 Chapter 8-Memories with Defects

fact that a column of length (2α - 1) has more than halve ones, and hence
there must be some overlap between the two specific columns;

Ad 2) if we combine 1) with 2), we see that any pair of bits is present in

some row and two arbitrary columns;

Ad 3) first take 2 different rows of length log2α . The columns all

have an equal number of zeros and ones. By deleting complementary pairs of
rows, except for the all zero and all ones row, the equal weight property
remains valid. We delete until 2α rows are left, with the property that all
columns have α ones. As this is part of the matrix construction, we place this
matrix in front of the original matrix by column permutations. Note, that
each row is uniquely specified by the first log2α bits.

These properties play an important role in the explanation of the coding

method. To be more specific, we consider the case where α = 3. The code
matrix has 6 rows and 10 columns.

0 0 0 0 0 0 0 0 0 0
0 0 1 0 1 1 0 1 1 1
C= 0 1 0 1 0 1 1 0 1 1 .
1 1 0 0 1 0 1 1 0 1
1 0 1 1 0 0 1 1 1 0
1 1 1 1 1 1 0 0 0 0

Any bit pattern of length 2 can be found in a row of length 10 and the first 3
digits uniquely specify each row of C.

Encoding The message x7 is represented by the vector

x10 = (0, 0, 0, x1, x2,··· , x7).

Hence, the vector x10 selects one out of 128 messages and the efficiency is
7/10.

Suppose that x 10 is not defect compatible in two of its components. The

defects are specified in location and value by d10 =(d1, d2, ··· , d10), where we
use a ? in case there is no defect at a particular location, otherwise we have
the symbol 0 or 1. Then, from the code matrix we take a row vector c10 (x10,
d10) that depends on x10 and d10 such that

151
 Chapter 8-Memories with Defects

c10 = x10  c10 (x10,d10)

is defect compatible. The constructed vector c10 is stored instead of x10.

Decoding
The decoding is done as follows. The vector x 10 has 3 all zero initial
components. Hence, the decoder (reader) knows which row of C is used in
order to make x10 defect compatible. This row is added modulo-2 to c10 and
the last 7 components of the result specify x7 again, for

c10  c10 (x10,d10) = x10 => x7

Example Suppose that we want to store the message (1, 1, 0, 1, 0, 0, 0) as

x10 = (0, 0, 0, 1, 1, 0, 1, 0, 0, 0).

The memory has a defect-1 in the 3rd position and a defect-0 in the 4th
position, i.e.

d10 = (?, ?, 1, 0, ?, ?, ?, ?, ?, ?).

The encoder adds modulo-2 the vector

c10 (x10,d10) = (1, 0, 1, 1, 0, 0, 1, 1, 1, 0)

to

x10 = (0, 0, 0, 1, 1, 0, 1, 0, 0, 0)

and stores

c10 = (1, 0, 1, 0, 1, 0, 0, 1, 1, 0).

The decoder sees as three initial components (1, 0, 1) and adds the vector

(1, 0, 1, 1, 0, 0, 1, 1, 1, 0)

to c10 which results in a decoded vector

x10 = (0, 0, 0, 1, 1, 0, 1, 0, 0, 0)

152
 Chapter 8-Memories with Defects

and message

x7 = (1, 1, 0, 1, 0, 0, 0).

The overall, close to optimal, efficiency is 7/10 at a defect rate of 2/10. One
problem is to show that this method is the best that we can have for 2 defects
in a word of length n.

8.2.4 A general method for t = pn defects in n positions

The previously described coding methods are special cases of the general
description by Kuznetsov and Tsybakov [64]. They show that there exists an
encoding matrix C that facilitates utilization of the fraction (1-p) of non-
defective memory cells, which is the best we can expect.

In the sequel, we assume that there are t defects in a length n binary vector
and the defect fraction is t/n = p. The encoding matrix C is given in Figure
8.4, where the first (n-k) columns are used to uniquely identify a particular
row (we have 2n-k rows). A particular row is used to make the vector x n
defect compatible.

n-k k

u defects in the first n-k

columns

v = t-u defects in the last

v k columns
2n-k

2n-k-u

xn = ( 0, 0, ···, 0 --------- xk --------- )

Figure 8.4 Non-linear encoding (Kuznetsov-Tsybakov)

153
 Chapter 8-Memories with Defects

Suppose that for a particular defect vector dn we have u defects in the first
(n-k) positions and v = t - u in the last k positions, respectively. Then, the
maximum number of rows that can be used to match the defect vector is
2n-k-u, because u positions are equal to the defects and thus fixed. If the
matrix is such that we have a pattern in the selected 2n-k-u rows that can be
used to match the defects in the last k positions, the matrix is called useful.
We calculate the fraction of matrices that is not useful and when this fraction
is smaller than 1, a useful matrix exists.

The v defects in the last k positions and the u defects in the first n-k
positions specify a matrix of dimension v × 2n-k-u. The number of matrices
where the given defect pattern does not occur is given by

n k u
f1  ( 2 v  1) 2 .

The remaining part of the 2n-k × k matrix can be chosen freely. The number
of choices f2 is

nk nku
f2  22 k  v2 .

For the particular defect vector, the fraction of useless matrices is thus given
by

n k u nk
k  v2 n  k  u
(2 v  1) 2  22
F  nk
22 k

 2
n k u
 1  2 v

nkt
 e2 .

Since we can have t defects with all possible combinations of defect-0 and
defect-1, the total fraction of matrices that cannot be used is upper bounded
by

n
F  exp(2 n  k  t  ln 2 t ).
t

154
 Chapter 8-Memories with Defects

If the efficiency is defined as R = k/n, then F is less than 1 for

t 1 n
R 1  log 2 (ln 2 t ).
n n t

Hence, for large values of n, the storage efficiency approaches

R => 1 - t/n = 1 - p.

This is a surprising result with many applications, such as writing in Write

Once Memory or Flash memory [68]. Since the defect matching as designed
by Kuznetsov-Tsybakov is a non-linear method, it might be useful to look at
the potential application of linear error correcting block codes and analyze
the performance.

8.3 Defects and Linear Codes

8.3.1 Defect matching

We start with the systematic encoding matrix Gk,n for code with minimum
distance dmin that has the form

Gk,n = [ Ik,k Hk,n-k ],

where Ik,k is the k × k identity matrix, and Hk,n-k the check part. The parity
check matrix for this encoder is given by

Hk,n-k
(Hn-k,n)T = .
In-k,n-k

Since the code has minimum distance dmin, any dmin - 1 rows of HTn,n-k
are linearly independent and thus in this case dmin - 1 columns of

Gn-k,n = Hn-k,n

= [ In-k,n-k (Hk,n-k)T ] .

155
 Chapter 8-Memories with Defects

We can use the systematic code Gn-k,n to construct any combination of dmin -
1 digits by some linear combination of rows. The matrix Gn-k,n is taken as
part of the bigger n × n matrix that has the structure as given in Figure 8.5,
where 0k,n-k is the all zero matrix with dimensions k and n-k. As before, the
vector xk and the defect vector dn = (d1, d2,···, dn) determine the vector cn-k
such that

cn = (cn-k,xk) Gn,n

is defect compatible with dn.

In-k,n-k (Hk,n-k)T
Gn,n =
0k,n-k Ik,k

Figure 8.5 The matrix Gn,n for the defect matching procedure.

Example For the encoder in Figure 8.6a, the first 4 rows are derived from a
systematic encoder with a minimum distance of 4 and code word length 7.
We can thus match 3 defects.

1 0 0 0 1 1 1
0 1 0 0 1 1 0
0 0 1 0 1 0 1
0 0 0 1 0 1 1
0 0 0 0 1 0 0
0 0 0 0 0 1 0
0 0 0 0 0 0 1

Figure 8.6a A 3 defect matching code generator , n = 7, k = 3

8.3.2 Defect matching and random error correction

Extension to defect matching and random error correction is done as follows.

We extend Gn,n with some additional columns such that the resulting matrix
is an encoding matrix for an error correcting code. At the same time the top
rows are used for defect matching. Hence, at reading we first correct the

156
 Chapter 8-Memories with Defects

random errors and then find back the information. The problem is to find the
optimum combination of defect matching and error correction. This problem
was tackled by Heegard [67]. We will illustrate the method by examples.

Example The encoding matrices in Figure 8.6b and Figure 8.6c follow
from the (7,4) Hamming code with minimum distance 3. In both cases we
have the same distance properties and thus the same error correcting
capabilities (1-error correcting).

In Figure 8.6b we use the first row to make the vector x7 defect compatible,
and in Figure 8.6c we use the first 3 rows, respectively.

1 1 1 1 1 1 1
0 1 0 0 1 1 0
0 0 1 0 1 0 1
0 0 0 1 0 1 1

Figure 8.6b 1-defect and 1-error correcting, n = 7, k = 3

1 0 0 0 1 1 1
0 1 0 1 1 0 1
0 0 1 1 1 1 0
0 0 0 1 0 1 1

Figure 8.6c 2-defects and 1-error correcting, n = 7, k = 1.

Note, that the storage efficiency is the same for the situation 8.6a and 8.6b.

8.4 Defects as Symbols

Instead of using a binary code generator matrix, we can use a generator
matrix of an RS code over GF(2m), where symbols represent m binary digits.
We consider symbols as defects instead of binary digits.

In Figure 8.7 we give a particular partitioning for an (n–δ) × n RS encoding

matrix Gn-δ,n with elements from GF(2m).

157
 Chapter 8-Memories with Defects

Properties:
- for the RS code generated by the encoding matrix Gn-δ,n, the
minimum distance is (δ + 1);
- any (n-k) × (n-k) sub-matrix of Gn-k,n has rank n-k.

Encoding
For an information vector xk- δ, we can construct cn-k such that

cn = (cn-k,xk-δ) Gn-δ,n

is symbol defect matching.

We receive

rn = cn  en.

Decoding
Using Gn-δ,n we can correct the random errors that might occur after storing
the code vector cn. After decoding the random errors, the vector (cn-k,xk-δ) can
be reconstructed and thus we can find back the original encoded information
xk-δ.

n-k Gn-k,n

< = Gn-δ,n
k-δ Gk-δ,n

Figure 8.7 Dimensions for the RS encoding matrix Gn- δ,n

Note that the error correcting properties appear when δ > 0. The defect
matching properties and the efficiency depend on the values of (n – k) and (k
– δ). For δ = 0, we store k symbols in n positions. The efficiency is

k nk
R  1  1  (defect fraction).
n n

158
 Chapter 8-Memories with Defects

We observe, that the RS code is optimum!

Remark. Although not very practical, the same defect matching procedure
can be designed for convolutional codes, see [65].

Application to the Write Once Memory (WOM)

Rivest and Shamir considered updating punch cards, punch tapes, and other
storage media which degrade from one updating to another one [69]. In this
section we show how to get the capacity of the punch card and other similar
degrading memories by additive coding. In order to be able to use a punch
card at least twice we must restrict the number of holes punched at the first
use of a new punch card by some number p, 0 < pn < n. Under this
restriction there are
pn
n
M   i 
i 0

different ways to punch a new card. Therefore, M messages can be

represented in this way, and during the first use of the punch card we store
log2M bits.

The configuration of holes on the punch card can be represented by the

binary vector dn, where di = 1 if the ith position of the card is punched (a
hole), and di = ? otherwise.

At the second use of the punch card the existing configuration of holes,
represented by the vector dn, can be used by a defect matching code. If we
have a defect matching code for a fraction p of defects, we can store a
maximum of (n-pn) bits in the second writing. Hence, the total normalized
asymptotic number of bits stored in two writings is

1 
pn
n
T  lim
n  n 
 
log 2    (n  pn)
i 
 i 0 
 h(p)  (1  p) bits/memory cell.

If the WOM is used three or more times, we can restrict the writing in every
step and optimize the fraction of holes to be punched [66].

159
 Chapter 8-Memories with Defects

8.5 Concluding Remarks

This chapter describes the development of coding for memories with defects.
The research was inspired by the researcher Alexander Kutznetsov, from the
Russian Academy of Sciences, IPPI, in Moscow. The defect model is used
today as a basic model in for instance coding for Write Once Memories and
also Flash Memories, see also [78]. We describe the defect model and
discuss the storage capacity for different modes of use. Defect matching, as
developed by Kutznetsov and Tsybakov, uses non-linear codes for making
information suitable to be stored in a memory where errors are permanent
(defects). Their general method achieves the theoretical capacity. As a
result, we can use a fraction (1-p) of a memory when a fraction p of the
memory is defective. A surprising result! The matching is generalized to
linear codes. When defects occur as symbols, RS codes obtain optimum
performance.

The concept of the content of the chapter is given in Figure 8.8

defect models

defect matching codes

defects and linear codes

defects as symbols

Figure 8.8 Concept of the content of Chapter 8

160
 Appendix

pend
App dix
A.1 A bit of In
nformattion Theoory
Shannonn entropy is a basic conceept in inform mation theoryy. For a sourrce with
output m
messages S ϵ {s1, s2, ···, sM}, cardinalitty M and pro
obabilities (p1, p2, ···,
pM), the entropy is deefined as

M
H(S)    p i log 2 p i bits. (A.1.1a)
(
i 1

Shannonn showed, thhat H(S) is th he minimum m average num mber of bits needed
to descriibe the sourcce output. The binary entrropy, h(p) is defined as

h(p) = -plogg2p – (1-p) log2(1-p) bits. (A.1.1b)

(

Figure A
A.1 gives the graphical reepresentationn of h(p).

h(p)

Figure A.1 The

F T graphicaal representattion of h(p)

161
 Appendix

The conditional entropy H(S | T) is the minimum average description length

of S given the side information T. It is defined as

H(S| T)    P(s, t)log2P(s | t) . (A.1.1c)

S, T

It follows automatically that H(S | T) ≤ H(S). The joint entropy,

H(S,T) = H(T) + H(S | T)

= H(S) + H(T | S).

The difference

I(S;T) = H(S) – H(S | T)

= H(T) – H(T | S),

is called the mutual information, and can be seen as the reduction in the
average description length of S given the observation T (or the other way
around). Conditional entropy, like H(S | T), is also called equivocation.

In data transmission we have a channel input ensemble X and a channel

output ensemble Y connected via the mutual information I(X;Y). The
maximum of I(X;Y) over the input probability distribution P(X) is called the
channel capacity

C  max I(X; Y)
P(X)
(A.1.2)
 max[H(X)  H(X | Y)].
P(X)

Channel codes exist that can transmit at a rate as close to capacity as wanted,
and at the same time achieve a decoding word error probability as close to
zero as required. To achieve this, the word length has to go to infinity.

162
 Appendix

Remark Recommended text books:

- T. Cover and J. Thomas, Elements of Information Theory 2nd

Edition, Wiley Series in Telecommunications and Signal Processing

- A classical book by R. Gallager is also highly recommended:

R Gallager, Information Theory and Reliable Communication

163
 Appendix

A.2 A Short Introduction to Galois fields

The purpose of this appendix is to give a brief introduction to Galois Fields
and it applications. We start with the system of calculations using the
numbers modulo p, where p is a prime number. Then we continue to do the
equivalent for symbols that can be considered as consisting out of m binary
digits.

A.2.1 Calculations modulo a prime number, GF(p)

Suppose that we do calculation modulo a prime number p.

Definition: The Galois Field GF(p) is the collection of p elements

{0, 1, 2, ···, p-1}.

Then, we use the following properties.

Property There exists a primitive element, called α such that all p-1 powers
of α are different modulo p and α p-1 = 1 modulo p.

Proof Suppose that αi = 1 modulo p, i < p – 1. Then α j-i α i = α j = α j-i and

thus not all elements are different, which contradicts the assumption.

Example let p = 7 and calculations modulo p. The 6 powers of the number 3

are: 31= 3, 32 = 2, 33 = 6, 34 = 4, 35 = 5, 36 = 1. They are all different.

Property Every element has an inverse.

Proof Since αp-1-i αi = 1 modulo p, αp-1-i is called the inverse of αi.

Furthermore, α j α i = αj+i modulo p.

Example Let p = 7 and calculations modulo p. The inverses are given by:
3-1= 5, 3-2 = 4, 3-3 = 6, 3-4 = 2, 3-5 = 3, and 3-6 = 1.

We thus have defined multiplication, inverse and the unit element 1. The
symbol 0 and all the nonzero elements together form the Galois Field GF(p).
We have an equivalent definition for a Galois Field using polynomials.

164
 Appendix

A.2.2 Calculations modulo a binary minimal polynomial

We consider polynomials of delay  0, i.e.

P(X) = p0 + p1 X + ••• + pm-1 Xm-1 , pi є {0,1}.

Polynomials can be added or multiplied, where the coefficients of equal

exponents are assumed to be added modulo 2.

Calculations modulo a polynomial g(X) can be interpreted as follows.

P(X) modulo g(X) = r(X) means that P(X) = a(X)g(X) + r(X), where the
degree{r(X)} < degree {g(X)}.

Example (1+X4) modulo(1+X+X2) →(1+X4) = ( X+X2) (1+X+ X2) + (1+X).

Definition A binary polynomial without divisors is called irreducible.

Definition An irreducible polynomial I(X) of degree m is called minimal if

and only if (iff)

I(X) divides (Xn + 1), where the smallest n is n = (2m -1).

Example (1 + X + X2) divides X3 + 1 and is minimal.

Example (1 + X + X3) divides X7 + 1 and is minimal.

Remark At this position we want to remark that there are 2m different

polynomials of degree less than m. With every polynomial we can uniquely
connect a binary m-tuple or symbol of length m.

Example The binary 4 tuple (0,1,0,1) can be represented as X + X3.

We use the following important equivalence notation using the minimal

polynomial I(X), i.e.

Xi modulo I(X) ≡ αi.

Definition The Galois Field GF(2m) is the collection of 2m elements

{0, 1, α, α2,···, αp-2}.

165
 Appendix

As a consequence, every element corresponds to a binary polynomial of

degree less than m and is equivalent to an m-tuple.

Property All the elements αi are different.

Proof Suppose that two elements are the same, then for i < j,

(Xi + Xj) modulo I(X) = Xi (Xj-i + 1) = 0 modulo I(X).

Since j - i < 2m -1 and I(X) minimal, this is impossible.

Property The element αp-1= 1.

Property αi αj = αi + j.

Property Since αp-1= 1, we can reduce the exponents modulo p-1 as before.

Property Every nonzero element has an inverse.

This follows easily from αi α-i = αi αp-1-i = αi αj.

Example Take I(X) = 1 + X + X3, which divides X7-1. The following table
lists the elements of the Galois Field

TABLE A.2.1

modulo binary inverse

1 + X + X3 3-tuple
____________________________________________

α X X (010) α6
α2 X2 X2 (001) α5
α3 X3 1+X (110) α4
α4 X4 X + X2 (011) α3
α5 X5 1 + X+ X2 (111) α2
α6 X6 1 + X2 (101) α1
α0 = 1 X7 1 (100) 1

Adding the element 0 gives the additional 3-tuple (000) and thus we have 8
elements. We can add, multiply and calculate an inverse with m-tuples
represented as elements in a Galois Field GF(2m).

166
 Appendix

A.3 Vandermonde’s Determinant

In the theory of RS codes, we need the property that any k × k submatrix
from the generator matrix Gk,n formed by k rows and any k columns has full
rank. We proof this by showing that the corresponding determinant is
unequal to zero.

Without loss of generality, we can consider the Vandermonde matrix

1 1  1
1 2  k
Vk = 12 22  k2 .

1k-1 2k-1  kk-1

Let P(k) be the proposition that Dk = ∏ 1 ≤ i < j ≤ k (i − j ) .

Proof by induction: P(1) is true, as this just says D1 = 1. P(2) holds, as it is

the case: D2 = (1 − 2 ) . This is our basis for the induction.

Now we need to show that, if P(t) is true, where t ≥ 2 , then it logically

follows that P(t+1) is true. So this is our induction hypothesis:

Dt = ∏ 1 ≤ i < j ≤ t (i −j ).

Then we need to show: Dt+1 = ∏ 1 ≤ i < j ≤ t+1 (i −j ).

This is our induction step: Consider the determinant:

1 1 1  1
x 2 3  t+1
Dt+1 = x2 22 32  t+12 .

xt 2t 3t  t+1t

167
 Appendix

If you use the Expansion Theorem for Determinants to expand it in terms of

the first column, you can see it is a polynomial in x whose degree is no
greater than t. Call that polynomial f(x).

If you substitute any i for x in the determinant, two of its columns will be
the same. So the value of such a determinant will be 0. Such a substitution in
the determinant is equivalent to substituting i for x in f(x). Thus it follows
that f(2) = f(3) = … = f(t+1) = 0 as well. So f(x) is divisible by each of
the factors 2, 3, …, t+1. All these factors are distinct otherwise the
original determinant is zero. So:

f(x)=C(x− 2)(x−3)⋯(x−t )(x−t+1) .

As the degree of f(x) is no greater than t , it follows that C is independent of

x. From the Expansion Theorem for Determinants, we can see that the
coefficient of xt is:

1 1  1
2 3  t+1
Dt+1 = 22 32  t+12
 .
t-1
2 3t-1  t+1 t-1

which by the induction hypothesis, this is equal to ∏ 2 ≤ i < j ≤ t+1 (i −j ).
So this has to be our value of C. Thus, we have:

f(x) = C(x − 2)(x − 3)⋯(x − t )(x − t+1) ∏ 2 ≤ i < j ≤ t+1(i −j ).

Substituting 1 for x , we retrieve the proposition P(t+1) . So P(t) ⟹

P(t+1) and the result follows by the Principle of Mathematical Induction.
Therefore:

Dk = ∏ 1 ≤ i < j ≤ k (i −j ),

which is unequal to zero since all elements i, i = 1,k are different. .

168
 Appendix

A.4 RS Encoding and Syndrome Former

We first describe the general encoding matrix as it is convenient for our
further considerations. We derive some of the RS code properties, like
distance and encoding matrix rank. These are based on the Vandermonde
matrix [A.3].

The encoder
The encoder performs the inner product of an information vector A of length
k with a k x n encoding matrix Gk,n. Both A and Gk,n have components from
GF(2m). The k x n encoding matrix consist of k consecutive rows of the
following matrix

1 1 1  1
1  2  n-1
Gn,n = 1 2 4  2(n-1) , (A.4.1)

1 n-1 2(n-1)  (n-1)(n-1)

where  is a primitive element of GF(2m ) and n = 2m -1.

Example The first k rows give as an encoding matrix

1 1 1  1
1  2  n-1
Gk,n = 1 2 4  2(n-1) . (A.4.2)

1 k-1 2(k-1)  (k-1)(n-1)

The following two properties are basic for the applications of RS codes.
Without loss of generality, we choose Gk,n, to be the first k rows from Gn,n as
in (A.4.2).

Property 1 Any k columns of Gk,n have rank k.

This follows easily from a particular selection of k columns

169
 Appendix

1 1  1
p q  r
V(p, q, , r ) = 2p 2q  2r .
k

p(k-1) q(k-1)  r(k-1)

which has the form of a Vandermonde matrix (note that all second row
elements are different).

Property 2 The minimum distance dmin of the code is n-k+1.

For linear codes, the minimum distance is equal to the minimum Hamming
weight of a nonzero code word. This property follows from the fact that any
k columns are linearly independent and thus no linear combination of rows
can give a code word with k zeros. Hence, the minimum nonzero Hamming
weight of any code word is n-k+1.

The same proof follows from the following observation (Jack van Lint). Let
A(X) be a polynomial over GF(2m) of degree k-1. We can evaluate this
polynomial for all n = 2m-1 possible different powers of the primitive
element  of the field GF(2m). This is equivalent to the inner product
encoding operation as described before. Since A(X) can have only k-1
different roots, at least n-k+1 evaluations must be nonzero and thus any
nonzero code word has Hamming weight larger or equal to n – k + 1. Since
the minimum distance for linear codes is less than or equal to n – k + 1, dmin
= n-k+1.

Example The (7,3) non-systematic R-S encoding matrix is given by

1 1 1 1 1 1 1
G= 1  2 3 4 5 6
1 2 4 6  3 5

170
 Appendix

In systematic form, we have

1 0 0 3 1 3 2
G= 0 1 0 5 1   .
0 0 1 4 1 6 4

The matrix Gn,n has an inverse, given by

1 1  1 1
1  n-1  2 
Hn,n = 1 2(n-1)  4 2 . (A.4.3)

1  (n-1) ( n-1)  2(n-1)  n-1

The syndrome former HT that corresponds to the encoder (A.4.2), has

dimension n  ( n-k) and is given by the last n-k columns from Hn,n, i.e.

1  1 1
 n-k  2 
HT = 2(n-k)  4 2 . (A.4.4)

 (n-1) ( n-k)  2(n-1)  n-1

Property 3 The product GHT = 0.

We recall that n = 2m -1. For GF(2m), we have  n = 1 and thus for  = q, q
 n-1,

 n – 1 = ( – 1) ( n-1 +  n-2 +  + 1 ) = 0.

Any inner product P of a row of Gk,n with a column of HT looks like

P = 1 +  i+j +  2(i+j) +  +  (n-1)(i+j)

= 1 +  +  2 +  +  n-1

= 0,

for 0 < i + j  n-1, and thus   1.

171
 Appendix

The syndrome former HT can be used to decode a received code word at the
receiver. We distinguish the following steps.

- Suppose that after transmission of cn = xkGk,n, the receiver obtains a

noisy version, rn = cn + en, where addition is in GF(q = 2m).

- The decoder calculates the product rnHT = sn-k, which is called the
syndrome. The syndrome only depends on the noise word en.

- For a code with minimum distance (n-k+1) all error words of weight
less than (n-k)/2 give a different syndrome. For, otherwise a vector of
weight less than n-k+1 can be constructed that gives a syndrome 0n-k.
According to the definition of the minimum distance, this is not possible.

- The number of different syndromes and thus correctable error words is

qn-k. The number of error words of weight (n-k)/2 or less is given by

 n k   n k 
 2   n k   2 
 n   n
  (q  1)i  q  2  .
i    q n  k ,
i
i 0   i 0  

where we assume that n = q-1. From this we can see, that for small k,
almost all syndromes correspond to an error word of weight  (n-k)/2.
For larger k, the bound will not be tight.

Example We give two examples: the single parity check (n,n-1) RS code
with efficiency k/n = (n-1)/n, minimum distance 2 and the repetition code
with efficiency 1/n and minimum distance n.

For the (n,n-1) RS code, we start with the generator matrix

1  2  n-1
1 2 4  2(n-1) .
Gn-1,n = 

1 n-1 2(n-1)  (n-1)(n-1)

172
 Appendix

The parity check matrix H is given by

H= [ 1, 1, , 1].

We give the equivalent (n-1)  n encoding matrix G in systematic form as

1 0 0  1
0 1 0  1
Gn-1,n = 0 0 1 0  1 . (A.4.5)

0 0  0 1 1

It is easy to see, that (A.4.5) generates all code words cn such the inner
product cn HT = xn-1Gn-1,n HT = 0n-k. This matrix Gn-1,n is the simplest possible
and simplifies the encoding procedure drastically.

Decoding for this code is in principle the same as for the SPC. After taking
hard decision, the syndrome gives the error value. The position can be
determined by finding the position that minimizes the Euclidean distance.

Example For the R = 1/n repetition code, the encoding matrix is

G1,n = [ 1, 1, ,  , 1]. (A.4.6)

The corresponding parity check matrix HT is given by

1  1 1
 n-1  2 
HT = 2(n-1)  4 2 . (A.4.7)

 (n-1) ( n-1)  2(n-1)  n-1

After hard decision, the decoding procedure is a simple majority vote. In this
case, more than n/2 errors lead to decoding decision errors.

Property 4 Any (n-k) rows of HT as given in (A.4.4) have rank n-k.

This follows directly from the minimum distance of the code generated by
G, which is n-k+1. Thus, all n-k or less rows of HT are linearly independent.
Furthermore, H is an RS encoding matrix with minimum distance k + 1.

173
 Appendix

Remark Suppose that we represent a code word cn of length n as a degree

(n-1) polynomial C(X). Then, cnHT = xkGHT = 0n-k is equivalent to the n-k
evaluations of C(X) for , 2 ,,  n-k. We can say that

C(X) = A(X)g(X) = A(X) (X-)(X-2) (X- n-k),

where g(X) is called the generator of the RS code and A(X) the information
polynomial of degree (k-1).

The calculation X n-k A(X) modulo g(x) gives a rest of degree less than n-k,
also called check part. Hence, we can have a systematic code word
containing the original information by writing C(X) = (X n-k A(X) + X n-k
A(X) modulo g(x)). Note that C(X) is still a multiple of g(X), since C(X)
modulo g(X) = 0.

Remark We developed the theory of syndrome decoding for convolutional

codes in [92,93,94]. This approach leads to lower complexity decoding when
statistical properties of the noise are considered. Furthermore, structure in
the encoding procedure can give rise to state-space reductions.

174
 Appendix

A.5 The Decoding Algorithm

Recall that the syndrome former has the following form

1  1 1
 n-k  2 
HT = 2(n-k)  4 2 .

 (n-1) ( n-k)  2(n-1) (n-1)

If we write the code words in the form of a polynomial, then the code word
C is written as C(X). In the same way, we write the received word as

R(X) = C(X) + E(X),

where E(X) represents the error symbols. Using the properties of the code
word C(X), we calculate the syndromes

Sj = R(X = αj+1) = C(X = αj+1 ) + E(X = αj+1 ), 0  j  2t-1

= E(X = αj+1)

n 1
=  E i (  i ) j 1
i0

t
=  E i k (  i k ) j 1 ,
k 1

where we have assumed t errors at position i1, i2, , it, 0  i  n-1. The
syndrome values Sj , 0  j  2t-1 define the syndrome polynomial


S(X) =  S jX j
j0

 t
=  X j  E i k ( i k ) j1
j 0 k 1

175
 Appendix

t
=  E i k ( i k ) /(1  X i k ).
k 1

The error locator polynomial for t errors at position ik, k=1,2, ···, t is defined
as

t
L(X) =  (1  X i k ).
k 1

The product L(X)S(X) can be written as

t t
L( X )S( X )   E i k ( ) 
ik
(1  X i  )  W ( X ) .
k 1 k

We remark that the degree of W(X) less than t. Actually, the polynomial
S(X) is known only through the coefficient of X2t-1 so that in terms of the
known syndrome components we can write the key equation

L(X)S(X) = W(X) modulo X2t.

The polynomials L(X) and W(X) are found according to Euclid’s algorithm.
This algorithm gives a unique valid solution in a finite number of steps (an
alternative is the Berlekamp Massey algorithm). We therefore reformulate
the key equation as

L(X)S(X) + F(X) X2t = W(X).

We have to find the solution L(X ) under the condition that the greatest
common divisor of S(X) and X2t is given by

gcd( S(X), X2t) = W(X).

For t errors, the polynomials L(X) and W(X) have degree t and degree < t,
respectively and are the only solution to the key equation and hence to the
problem.

We use the polynomial L(X) for finding the error locations. For an error at
position im, according to the definition,

176
 Appendix

t
L (  i m )   (1    i m  i k )  0.
k 1

The search for the positoins where L(*) = 0, is called the Chien search.
Knowing the incorrect positions, also gives at least k correct positions and
thus, this allows us to reconstruct the encoded information. An interesting
topic that remains, is the complexity of this procedure.
Example
We use the GF(23) from Table A.2.1 and a (7,5) RS code defined as
C(X) = A(X)g(X) = A(X) (X-)(X-2).
For an information polynomial
A(X) =  + 3X4,
The code word polynomial becomes
C(X) = 4 + 5X +  X2 + 6X4 + X5 + 3X6.
We receive

R(X) = C(X) + E(X) = 4 + 5X+ X2 + 6X4 + 6X5 + 3X6.

The syndrome

S0 = R() = 4 + 6 + 3 + 3 + 4 + 2 = 1 = Eii ,

S1 = R(2) = 4 + 7 + 5 + 1 + 2 +  = 5.

Thus, S(X) = 1 + 5X. for t = 1, we have to find the gcd (1 + 5X, X2).
From this, we find

(1 + 5X)(1 + 5X) + 3X2 = 1.

The error location is at position 5. The error value is S0 -5 = 2.

177
 Appendix

A.6 Middleton Class-A Impulse Noise Model

The Class-A noise model is a model for noise that is impulsive in
combination with Gaussian noise. A Gaussian noise component is added to
model the (almost) always present thermal receiver and background noise.

N(0,  G2 ) n N(0, σ )

background impulse

in out

Figure A2 Representation of the Middleton model

The model, Figure A2, is characterized by:

- σ , the variance of the Gaussian distributed background noise;

- σ is the average variance of the Gaussian distributed impulse noise;
- The parameter A is called impulse index and is given by the product of
the average number of impulses per unit time and the mean duration of
the emitted impulses entering the receiver. For small A the noise has a
structured/impulsive character;
- The parameter T gives the impulse strength or ratio between the mean
power of the Gaussian and the mean power of the impulsive noise
2 2
component, i.e. T:= σG /σ I ;
- The state m of the model is selected according to the Class-A noise
probability density function [25],

Am
Pm  e  A ; m  0.
m!

The impulse noise probability distribution is given by

178
 Appendix

n2
 
1 2 2m
p( n | m )  e , p( n )   Pm p(n | m) .
2 2m m 0

The average noise variance σ 2  σ 2I  σ G

2
. Note that for m = 0, we only have
background noise.

Example A simple two-state model can be defined as follows:

P(m = 0) = 1- P(m = 1) = 1 – A;

σ 2  σ 2I  σ G
2 2 2
; T:= σ G /σ I .

The impulse noise channel can be seen as an infinite number of parallel

channels each with specific probability density p(n | m), where before
transmission, one of them is selected with probability Pm. The Class-A
Middleton channel model is depicted in Figure A3. Note that the two models
from Figure A2 and Figure A3 are fully equivalent.

AWGN channel 0

input X AWGN channel 1 output Y

··· ···
AWGN channel k

Am m
Pm  e  A ;  2m   2I 2
 G
m! A

Figure A3 Alternative Class-A Middleton channel model

Example As an example, for an impulse duration of 10-4 seconds and on the

average 100 impulses per second, we have A = 10-2. For a symbol
transmission rate of 104 symbols/second, the probability that a symbol is hit
by impulse noise is 10-2. It can be seen that the impulse duration plays an

179
 Appendix

important role in the transmission scheme and it is therefore very important

to have reliable measurements. For instance, for a transmission rate of 105
symbols/sec and an impulse width of about 10-4 seconds, about 10 symbols
are hit by the impulse noise.

Example In Table 4 we give an example of the noise power spectral density

for A=0.01 and impulse noise variance σ 2I  100 σ G
2.
From a communication
point of view, we may say that the impulse noise is 20 dB above the
background noise. Note that the probability of selecting a particular channel
depends on A. Channel m = 0 is the most likely one. For larger A, channels
with a higher number become more likely. The Class-A Middleton channel
model is in principle a memory-less channel model.

Table 4

Example of the noise power spectral density for

2 2
A = 0.01 and T = σG /σ I  0.01

Am m
Channel state Pm  e  A  2m   2I 2
 G
m! A

2
m=0 0.99 G
2
m=1 0.01 10 4   G
2
+ G
2
m=2 5 10-5 2  10 4  G
2
+ G

180
 Appendix

A.7 Impulse Noise Channel Capacity

Suppose that we have the two-state impulse noise channel model as given in
Figure A4. The channel has two states with additive Gaussian noise:

2
Gaussian: probability 1-A, variance σ G ;
2
Impulsive: probability A, variance  G   2I / A .

The average variance is σ 2  σ 2I  σ G

2
. The average input power spectral
density is P/2B.

2
1-A G

input X output Y

2  σ 2 /A
σG
A I

Figure A4 Simple impulse noise model

We can distinguish four different situations for the communication,

depending on whether the transmitter and the receiver are informed about the
state of the channel:

1. State known at transmitter and receiver

In this case, the transmitter can use a water-filling argument to optimize his
transmission. As a result, the input power spectral density is

PA = P/2B +  2I   2I / A ,

181
 Appendix

P1-A= P/2B +  2I ,

1 A 2
for P/2B  σI .
A

The channel capacity is given by

P 2 ) – AH( σ 2  σ 2 /A )bits
C+,+ = H(  σ 2I  σ G
2
) – (1-A)H( σ G G I
2B

 P   

2
 σG  σ 2I   2 
σG
= Blog 2  2B   ABlog 2   bits/second (A7.1)
 2   
 σG   σ G
2 σ 2I 
  
 A 

1 A 2
For P/2B <  I , only the Gaussian channel is considered and thus,
A

 P 2 
  σG 
2B(1  A)
C+,+ = (1-A) H  bits/transmission
 2
σG 
 
 

 P 2 
  σG 
2B(1  A)
= (1-A) Blog 2   bits/second (A7.2)
 2
σG 
 
 

Remark We see that in the high power region, water-filling is the same as
fully randomizing the channel for the output entropy H(Y). For low power,
we have an improvement.

2. State not known at transmitter

In this case, the transmitter always uses the same average input power since
it has no information on the channel state. The receiver has the information,
and thus

182
 Appendix

2 2
C-,+ = (1-A){H(P/2B +  G ) – H(  G )} +

 P  2
 2 σ 2I   2 σI 
+ A{H  σ  σ
– H G  }bits/transmission
A 
 2B G
 A  
 
(A7.3)
 P 2 
 P 
σ 2I
  G   2
 σG  
= (1  A)B log 2  2 B   AB log 2  2B A  bits/second.
 G 2  2 
   σ G2 σI 
  
 A 

3. State not known at transmitter and receiver

For this situation, we use a randomization or orthonormal transform (FFT) at
the transmitter and the receiver, see Figure A5. The output Y is assumed to

noise

Input X output Y
T-1 T

Figure A5 Randomized transmission scheme with transform T

be Gaussian with variance P/2B + σ 2I  σ G

2
. The capacity can then be
estimated as

P
C-,- = H(  σ 2I  σ G
2
) – H( σ 2I  σ G
2
) bits/transmission
2B
 P 2 
  σG  σ 2I 
= Blog 2  2B  bits/second. (A7.4)
 σG 2
 σ 2 
 I 
 

183
 Appendix

4. State known at transmitter, but not at receiver

We use the water-filling argument for the power when the channel state is
known. Since H(Y | X)  H(Y | X, state), we can estimate the channel
capacity as

C+,- ≤ C+,+

A better approximation of the C+,+ has to be found.

Conclusion

It can be seen from (A7.2) and (A7.3), that C-,+ ≈ C+,+. Hence, this shows that
an informed receiver can be as effective as an informed transmitter. In
principle, no feedback is needed.

The difference Δ between informed and un-informed transmitter and

receiver is defined as

 σ2 
 G 
Δ = 10log10   dB.
 σ2  σ2
 G I 

For σ 2I  100 σ G
2
, the gain in capacity by the channel state information is
about 20 dB.

For more information on impulsive noise mitigation and capacity bounds we

refer to the work by Jürgen Häring1 in [75,76,77,86,87].
1
Jürgen Häring, PhD 2001, University Duisburg-Essen, Germany, “Error Tolerant
Communication over the Compound Channel”

184
 Appendix

A.8 Channel Capacity and Water-filling

A.8.1 Gaussian Channel Capacity

Before we consider the impulse noise channel, we repeat the channel

capacity for the Additive White Gaussian Noise Channel with input X and
output Y, under the following conditions:

- channel bandwidth: 2B (double sided);

- input power spectral density: σ 2X  P / 2 B ;
2
- noise power spectral density: σ G ;
2
- entropy of the Gaussian noise: H(noise) = ½ log2(2πe σ G )bits.

2
N(oise) N(0, σ G )

X Y=X+N

Figure A6 Additive White Gaussian Noise Channel model

The capacity for the AWGN channel is

CAWGN = H(Y) – H(Y | X)

= H(Y) – H(noise)

 σ2  1  2 2 
=
1
log 2  Y   log  σ X  σ G  bits/transmission.
 σ2  2 2 2 
2  G   σG 

Assuming that the channel has bandwidth B, we can represent both the input
and the output by samples taken 1/2B seconds apart. If the channel noise is
2
Gaussian with power spectral density σ G , each of the 2B noise samples per
2
second has variance σ G . The power spectral density per sample is P/2B and
thus, the channel capacity in bits/ per second is ,

185
 Appendix

 P 
C = Blog 2 1   bits/second.
 2Bσ 2 
 G 

A.8.2 Capacity of the Degraded Broadcast Channel

The broadcast channel (γ2 > σ2 and ) is depicted in Figure A7.

N(0, σ2) N(0, γ2)

n1 n2

Y Z

Figure A7 The degraded broadcast channel

We can represent the transmission from X to Z and X to Y in Figure A8,

where X = U + V, U and V Gaussian input variables with average power αP
and (1-α)P, respectively. The message U is for receiver Y and the message V
is for user Z. Since γ2 > σ2, the first user is more powerful and can thus also
decode U and subtract its influence from X. A simple AWGN channel
U→Y remains. The second user experiences the signal U and n2 as additive
white Gaussian noise and thus the achievable rate from transmitter to both
the receivers is

1  P 
C1 = log 2 1  ,
2  2 
(A8.1)
1   P 
C2 = log 2 1  .
2  P   2 
 

What remains is to proof that this is indeed the capacity region for the
degraded broadcast channel. For this, we refer to [20].

186
 Appendix

N1(oise): N(0, σ2)

V Y = V + N1

V: N(0, αP)

V: N(0, αP) N2(oise): N(0, γ2)

U Z = V + U + N2

U: N(0, (1- α)P)

Figure A8 The degraded broadcast channel where X = U + V

A.8.3 Water-filling for two parallel channels

Suppose that we have an AWGN channel with bandwidth B, divided into

two channels in portions αB and (1-α)B, respectively. The maximum
average power P = P1 + P2, where P1 and P2 are the average power for
channel 1 and channel 2, respectively. In addition, one channel has noise
N1 with variance γ2 and the other channel has noise N2 with variance σ2, γ2
> σ2 . The capacities in bits/second are

 P1 

C1  α B log 2 1  , (A8.2a)
 2α B γ 2 
 

 P  P1 
 
C 2  (1  )B log 2 1  . (A8.2b)
 2(1  ) B σ 2 
 

We maximize C1 + C2 as a function of P1, and find that for the maximum,

P1 = α ( P - B(γ2 - σ2) α), (A8.3a)

187
 Appendix

P2 = α ( P + B(γ2 - σ2) α) (A8.3b)

for P  (γ2 - σ2) α , otherwise P1 = 0. Figure A9 gives a graphical

interpretation.

P1/2αB P2/2α B

γ2

Power
Spectral
Density σ2

αB αB frequency

Figure A9 Graphical presentation of the water-filling argument

Note that P1 /2α B + γ2 = P2 2/α B + σ2. If the frequency band is divided

into more parts, a more complicated optimization algorithm is necessary, see
[22].

A.8.4 Water-filling for n parallel channels

For n parallel channels, each with the same bandwidth B, the sum capacity in
bits/second is given by

n  P 
CB  log 2 1  2Bi 2  , (A8.4.1)
i 1  i 

n
 Pi  P ,
i 1

188
 Appendix

where for every sub-channel the capacity is optimized by having a Gaussian

input distribution with power constraint Pi  0 subject to the sum power
constraint P. To maximize (A8.4.1) one uses the technique of Lagrange
multipliers. The Lagrange multiplier is given by

n  Pi  n
J(P1, P2, ···, Pn) = 2B log 2 1 
 2B 2  
 (P  Pi ) . (A8.4.2)
i 1  i  i 1

Differentiating with respect to Pi gives

1 P
 i   i2 .
2 ln 2 2B

The value of λ can be solved by using the sum power constraint P, where for
physical reasons Pi  0. See Figure A10 for a graphical interpretation. For
the left situation, enough power for all channels to have a positive capacity.

1
PSD
2 ln 2
P1/2B P1=0
12
1
P2/2B P2/2B
2 ln 2
 22
P3/2B P3/2B
32

B B B B B B

Figure A10 Graphical interpretation of the water-filling argument

for n parallel channels. Left: all channels are used

189
Appendix

190
 Appendix

10 References
[1] N. Abramson, “The throughput of packet broadcasting channels,” IEEE
Transactions on Communications, vol. COM-25, pp. 117–128, Jan. 1977
[2] Christof Haslach and A.J. Han Vinck, “A Decoding Algorithm with Restrictions
for Array Codes,” IEEE Transactions on Information Theory, pp. 2339-2345, Nov.
1999
[3] Christof Haslach and A.J. Han Vinck, “Correction to a Decoding Algorithm with
Restrictions for Array Codes," IEEE Transactions on Information Theory, pp. 479,
Jan. 2001
[4] A. Kuznetsov, F. Swarts, A.J. Han Vinck and H.C. Ferreira, "On the Undetected
Error Probability of Linear Block Codes on Channels with Memory," IEEE
Transactions on Information Theory, Vol. 42, No. 1, pp. 303-309, Jan. 1996
[5] A.J. Han Vinck and A.J. van Wijngaarden, “On Synchronization for Burst
Transmission,” IEICE, pp. 2130-2135, Nov. 1997
[6] Hiroyoshi Morita, Adriaan J. van Wijngaarden and A.J. Han Vinck, ”On the
Construction of Maximal Prefix-Synchronized Codes,” IEEE Transactions on
Information Theory, pp. 2158-2166, Nov. 1996
[7] J.J. Metzner and E.J. Kapturowski, “A general decoding technique applicable to
replicated file disagreement location and concatenated code decoding,” IEEE
Transactions on Information Theory, vol. 36, pp. 911–917, July 1990
[8] Shin-Chun Chang and J.K. Wolf, "On the T-User M-Frequency Noiseless
Multiple-Access Channel with and without Intensity Information," IEEE
Transactions on Information Theory, vol. IT-27, pp. 41-48, Jan. 1981
[9] A.J. Han Vinck and Jeroen Keuning, “On the Capacity of the Asynchronous T-
user M-frequency Noiseless Multiple Access Channel without Intensity
Information,“ IEEE Transactions on Information Theory, Nov. 1996
[10] A. Cohen, J. Heller and A. Viterbi, "A New Coding Technique for
Asynchronous Multiple Access Communication," IEEE Transactions on
Communications, vol. COM-19, pp. 849-855, Oct. 1971
[11] W.H. Kautz and R.C. Singleton, “Nonrandom Binary Superimposed Codes,“
IEEE Transactions on Information Theory, IT-10, pp. 363-377, Oct. 1964,
[12] A. J. Han Vinck, Samvel Martirossian and Peter Gober, “On q-ary
Superimposed Codes,” 1999 IEEE ITW, Kruger National Park, South Africa, June
20 - 25
[13] E.L. Titlebaum, “Time Frequency Hop Signals,” IEEE Transactions on
Aerospace and Electronic Systems, Vol. AES-17, pp. 490-494, July 1981

191
 Appendix

[14] H. Ferreira, L. Lampe, J. Newbury and T. Swart, editors, Power line

communications, John Wiley & Sons, 2000
[15] O. Hooijen, ”On the relation between network-topology and power line signal
attenuation,” In: International Symposium on Power Line Communication and its
Applications, pp. 45–55, March 1998
[16] O. Hooijen, ”A channel model for the residential power circuit used as a digital
communications medium,” IEEE Transactions on Electromagnic Compatibility,
40(4):331–6, 1998
[17] V. Balakirsky and A.J. Han Vinck, ”Potential performance of PLC systems
composed of several communication links,” In: International Symposium on Power
Line Communication and its Applications, p. 12–6, March 2005
[18] Y. Luo, A.J. Han Vinck and Yanling Chen, ”On the optimum distance profile
about linear block code,” IEEE Transactions on Information Theory 2010;56:1007–
14
[19] Lutz Lampe and A.J. Han Vinck, ”On Cooperative Coding for Narrowband
PLC Networks,” AEÜ(2011), doi: 10.1016/j.aeue., 2011
[20] P. P. Bergmans, “A simple converse for broadcast channels with additive white
Gaussian noise,” IEEE Transactions on Information Theory, Vol. IT-20, pp. 279-
280, 1974
[21] R.J. McEliece and L. Swanson, “A note on the wide-band Gaussian broadcast
channel,” IEEE Transactions on Communications, Vol. Com 35, pp. 452-453, 1987
[22] T.M. Cover and Joy Thomas, Elements of Information Theory, Wiley Series in
Telecommunications and Signal Processing
[23] N. Jindal, S. Vishwanath and A. Goldsmith, “On the Duality of Gaussian
Multiple-Access and Broadcast Channels,” IEEE Transactions on Information
Theory, Vol. 50, No. 5, pp. 768-783, May 2004
[24] T. Schaub, “Spread Frequency Shift Keying,” IEEE Transactions on
Communications, Vol. 24, pp. 1056-1064, 1994
[25] Morgan H.L.Chan and Robert W. Donaldson, “Amplitude, Width, and
Interarrival Distributions for Noise Impulses on Intra Building Power Line
Communication Networks,” IEEE Transactions on Electromagnetic Compatibility,
Vol. 31, Aug. 1989, pp. 320-323
[26] John Proakis, Digital Communications, McGraw-Hill, 1989
[27] J.A.C. Bingham, “Multicarrier Modulation for Data Transmission: An Idea
whose Time has Come,” IEEE Communications Magazine, pp. 5-14, May 1990
[28] Ian F. Blake, “Permutation Codes for Discrete Channels,” IEEE Transactions
on Information Theory, pp. 138-140, Jan. 1974
[29] M. Deza and S. A. Vanstone, “Bounds for permutation arrays,” J. Statist.
Planning Inference, vol. 2, no. 2, pp. 197–209, 1978.
[30] A.M. Michelson and A.H. Levesque, Error Control Techniques for Digital
Communications (John Wiley, 1985).
[31] R. Schweikert and A.J. Han Vinck, “A Convolutional Decoding Structure for
High Data Rate Applications,” IEEE Transactions on Communications, Vol. 39,No.
1, pp. 4-7, Jan. 1991

192
 Appendix

[32] K.A. Schouhamer Immink, Codes for Mass Data Storage Systems, Second fully
revised edition, Shannon Foundation Publishers, Eindhoven, The Netherlands, ISBN
90-74249-27-2, Nov. 2004
[33] G.D. Forney, Jr, Concatenated Codes, MIT Press, Cambridge, MA, (1966)
[34] P. Elias, “Error-free coding,” IEEE Transactions on Information Theory, pp.
29-37, Sept. 1954
[35] A. D. Wyner, “The wiretap channel,” Bell Sys. Tech. J.; 54, pp. 1355-1387,
1975
[36] L.H. Ozarow and A.D. Wyner, “Wiretap channel II,” Proc. Eurocrypt 84, A
workshop on Advances in Cryptology: Theory and Application of Cryptographic
Techniques; pp. 33-51, 1984
[37] G. David Forney, Jr., “Dimension/length profiles and trellis complexity of
linear block codes,” IEEE Transactions on Information Theory, 40(6): pp. 439-441,
Nov. 1994
[38] Y. Luo, C. Mitrpant, A.J. Han Vinck and K. Chen, “Some New Characters on
the WireTap Channel of Type II,” IEEE Transactions on Information Theory, pp.
1222-1228, ISSN 0018-9448, March 2005,
[39] C. Mitrpant, A.J. Han Vinck and Yuan Luo, “An Achievable Region for the
Gaussian Wiretap Channel with Side Information,” IEEE Transactions on
Information Theory, pp. 1222-1228, May 2006
[40] S. K. Leung-Yan-Cheong, “On a special class of wiretap channels,” IEEE
Transactions on Information Theory, vol. IT-23(5), pp. 625-627, Sept. 1977
[41] A. Juels and M. Wattenberg, “A Fuzzy Commitment Scheme,” ACM
Conference on Computer and Communications Security, pp. 28–36, 1999,
Lausanne, Switzerland
[42] S. Lin and D. J. Costello, Jr., Error Control Coding: Fundamentals and.
Applications, 2nd ed., Prentice Hall, 2004
[43] Y. Dodis, L. Reyzin and A. Smith, “Fuzzy Extractors,” Advances in
Cryptology, Springer Verlag, pp 523-540, Eurocrypt 2004
[44] Ulrike Korte et al., “A cryptographic biometric authentication system based on
genetic fingerprints,” Lecture Notes in Informatics,” pp. 263-276, Bonner Köllen
Verlag (2008)
[45] Y. Chen and A.J. Han Vinck, ”Look into the Biometric Authentication
Scheme,” accepted for publication in Secrypt 2011, Sevilla, Spain
[46] T. Ignatenko and F.M.J. Willems, ”Information Leakage in Fuzzy
Commitment Schemes,” IEEE Transactions on Information Forensics and Security,
pp. 337-348, June 2010
[47] A. Juels and M. Sudan, “A Fuzzy Vault Scheme,” Designs, Codes and
Cryptography, pp. 237-257, Febr. 2006
[48] K. Nandakumar, A.K. Jain and S. Pankanti, “Fingerprint-Based Fuzzy Vault:
Implementation and Performance,” IEEE Transactions on Information Forensics
and Security, pp. 744-757, Dec. 2007
[49] V. B. Balakirsky and A. J. Han Vinck, “A Simple Scheme for Constructing
Fault-Tolerant Passwords from Biometric Data,” Eurasip Journal on Information
Security, Volume 2010 (2010), Article ID 819376, 11 pages

193
 Appendix

[50] A.J: Han Vinck, R. Schweikert and H.P. Förster, “Message dependent
synchronization,” Global Telecommunications Conference, GLOBECOM’88, 1183
– 1187, 1988
[51] C.E. Shannon, "Communication Theory of Secrecy Systems," Bell System
Technical Journal, vol. 28(4), page 656–715, 1949
[52] A. J. Han Vinck and Y. Luo, “Optimum distance profiles of linear block
codes,” in Proc. IEEE Int. Symp. Inf. Theory, pp. 1958- 1962, July 2008
[53] Yanling Chen and A.J. Han Vinck, “A Lower Bound on the Optimum Distance
Profiles of the Second-Order Reed–Muller Codes,” IEEE Transactions on
Information Theory, pp. 4309 – 4320, Sept. 2010
[54] Rolf Johannesson, “Some long rate one-half binary convolutional codes with an
optimum distance profile,” IEEE Transactions on Information Theory, pp. 629-631,
1976
[55] H. Holma and A. Toskala, WCDMA for UMTS-HSPA Evolution and LTE, 4th
ed. London, U.K.: Wiley, 2007
[56] R. Tanner and J. Woodard, WCDMA-Requirements and Practical, Design,
London, U.K.: Wiley, 2004
[57] G. Solomon, “A Note on Alphabet Codes and Fields of Computation,”
Information and Control,” pp. 395-398, 1974
[58] Gottfried Ungerböck, “Channel Coding with Multilevel/Phase Signals,” IEEE
Transactions on Information Theory, Vol. IT 28, No. 1, pp 55-67, Jan 1982
[59] H. Hollmann, “Bounded-delay-encodable, block-decodable codes for
constrained systems,” IEEE Transactions on Information Theory, Vol. 6, pp. 1957-
1970, Nov. 1996
[60] Anil Mengi, Ph. D. Thesis, “On Combined Coding and Modulation,”
University Duisburg-Essen, Essen, Germany, 2010,
[61] Anil Mengi and A.J. Han Vinck, “Coded modulation with a constraint on the
minimum channel symbol duration,” Proceedings International Symposium on
Information Theory, ISIT 2009, Seoul, South Korea, pp. 1544-1548, June 2009
[62] A.J. Viterbi, “A rubust ratio-threshold technique to mitigate tone and partial
band jamming in coded MFSK systems,” in Proc. 1982 IEEE MILCOM, Boston,
MA, USA, Oct. 17-20, pp. 22.4.1- 22.4.5, 1982
[63] D.J.J. Versfeld, A.J. Han Vinck, J.N. Ridley and H.C. Ferreira, “Constructing
Coset Codes With Optimal Same-Symbol Weight for Detecting Narrowband
Interference in M-FSK Systems,” IEEE Transactions on Information Theory, pp.
6347-6353, Dec. 2010
[64] A. V. Kuznetsov, B. S. Tsybakov, “Coding in a Memory with Defective Cells,”
Probl. Peredachi Inf., 10:2, pp. 52–60, 1974
[65] Martin Borden and A.J. Han Vinck, ”On Coding for Stuck-at Defects,” IEEE
Transactions on Information Theory, pp. 729-735, Sept. 1987
[66] A.V. Kuznetsov and A.J. Han Vinck, "On the General Defect Channel with
Informed Encoder and Capacities of Some Constrained Channels,” IEEE
Transactions on Information Theory, Vol. 40, pp. 1866-1871, Nov. 1994,
[67] Chris Heegard, “Partitioned Linear Block Codes for Computer Memory with
Stuck – at Defects," IEEE Transactions on Information Theory, Vol. IT29, No.6, pp.
831-842, Nov. 1983

194
 Appendix

[68] Fangwei Fu and A.J. Han Vinck, “On the Capacity of Generalized Write-Once
Memory with State Transitions Described by an Arbitrary Directed Acyclic Graph,”
IEEE Transactions on Information Theory, Vol. 45, no.1, pp. 308-312, Jan. 1999
[69] R. L. Rivest and A. Shamir, “How to reuse the write once memory,” Inform.
Control, vol. 55, pp. 1-19, Oct.-Dec. 1982
[70] http://mathworld.wolfram.com/VandermondeDeterminant.html
[71] Yuan Luo, A.J. Han Vinck and Yanling Chen, “On the Optimum Distance
Profile about Linear Block Codes,” IEEE Transactions on Information Theory, Vol
56, pp. 1007-1014, March 2010
[72] R. Gallager, Information Theory and Reliable Communication, 1968
[73] Tadashi Wadayama and A.J. Han Vinck, “DC-Free Convolutional Codes,”
IEEE Transactions on Information theory, Vol. 48, pp. 162-173, ISSN 0018-9448,
Jan. 2002
[74] Tadashi Wadayama and A.J. Han Vinck, “A Multilevel Construction of
Permutation Codes,” IEICE Transactions on Fundamentals, pp. 2518-2522, Oct.
2001
[75] Jurgen Haering and A. J. Han Vinck, “Coding and Signal Space Diversity for
a Class of Fading and Impulsive Noise Channels,” IEEE Transactions on
Information Theory, pp. 887-895, May 2004
[76] Jurgen Haering and A. J. Han Vinck, ”IEEE Transactions on Information
Theory, pp. 1251-1260, May 2003
[77] Jurgen Haering and A. J. Han Vinck, “Performance Bounds for Optimum and
Suboptimum Reception under Class-A Impulsive Noise,” IEEE Transactions on
Communications, pp. 1130-1136, July 2002
[78] Frans Willems and A.J. Han Vinck, “Repeated Recording for an Optical Disk,”
7th Symposium on Information Theory in the Benelux, Noordwijkerhout, The
Netherlands, pp. 49 – 53, May 22-23, 1986
[79] Martin Vinck, Francesco P. Battaglia, Vladimir B. Balakirsky, A. J. Han Vinck,
and Cyriel M. A. Pennartz, ”Estimation of the entropy based on its polynomial
representation,” Phys. Rev. E 85, 051139 (2012) [9 pages]
[80] T. Ericson and V. Levenshtein, “Superimposed Codes in the Hamming Space,”
IEEE Transactions on Information Theory, pp. 1882-1893, Nov. 1994
[81] G. Poltyrev and J. Snyders, “Linear Codes for the Sum-2 Multiple-Access
Channel with Restricted Access,” IEEE Transactions on Information Theory, pp.
794-799, May 1995
[82] A.J. Han Vinck, “Capacity Achieving Coding for the “OR” Multiple Access
Channel,” ITW 1998, Killarney, Ireland , p. 127, June 22-26
[83] Peter Gober, Aspekte des unkoordinierten Mehrfachzugriffs, Verlag
dissertation.de, 2000
[84] V.I. Levenshtein and A.J. Han Vinck, “Perfect (d,k)-Codes Capable of
Correcting Single Peak-Shifts," IEEE Transactions on Information Theory, Vol. 39,
No. 2, pp. 656-662, March 1993
[85] Young Gil Kim and A.J. Han Vinck, “A New Collision Arbitration Algorithm
for FM0 Code in RFID Applications, ”International Conference on Multimedia and
Ubiquitous Engineering, MUE '07, pp. 992–994, 26-28 April 2007

195
 Appendix

[86] Victor N. Papilaya, Thokozani Shongwe, A. J. Han Vinck and Hendrik C.

Ferreira, "Selected Subcarriers QPSK-OFDM Transmission Schemes to Combat
Frequency Disturbances," 2012 IEEE ISPLC, Bejing, 2012, 200 – 205, 2012
[87] Der-Feng Tseng, Han, Y.S., Wai Ho Mow, Li-Chung Chang and Vinck, A.J.H,
“Robust Clipping for OFDM Transmissions over Memoryless Impulsive Noise
Channels,” Communications Letters, IEEE Volume: 16 , Issue: 7, pp. 1110 – 1113,
2012
[88] N. Pavlidou, A.J. Han Vinck, J. Yazdani and B. Honary, “Power Lines
Communications: State of the Art and Future Trends,” IEEE Communications
Magazine, pp. 34-40, April 2003
[89] Bin Dai, A.J. Han Vinck, Yuan Luo and Zhuojun Zhuang, “Capacity region of
non-degraded wiretap channel with noiseless feedback,“ Proceedings (ISIT), 2012
IEEE International Symposium on Information Theory, pp. 244-248, 2012
[90] A.J. Han Vinck, “Coding Techniques and the Two-Access Channel,” In
Multiple Access Channels: Theory and Practice Eds. E. Biglieri, L. Györfi, pp. 273-
286, IOS Press, ISBN 978-1-58603-728-4, 2007
[91] A.J. Han Vinck, W. Hoeks and Karel Post, “On the capacity of the Two-User
M-ary Multiple Access Channel with Feedback,” IEEE Transactions on Information
Theory, pp. 540-543 , July 1985
[92] J.P.M. Schalkwijk, A.J. Han Vinck and Karel Post, “Syndrome Decoding of
binary rate k/n convolutional codes,” IEEE Transactions on Information Theory, pp.
553 – 562, Sept. 1977
[93] A.J. Han Vinck, ”A low complexity stack decoder for a class of binary
rate (n-1)/n convolutional codes,” IEEE Transactions on Communications,
pp. 476-479, April 1984
[94] A.J. Han Vinck, A.J. de Paepe and J.P.M. Schalkwijk, ”A class of
binary rate one-half convolutional codes that allows an improved stack
decoder,” IEEE Transactions on Information Theory, pp. 389-392, July 1980
[95] H.C. Ferreira, A.J. Han Vinck, T.G. Swart and I. de Beer, “Permutation Trellis
Codes,” IEEE Transactions on Communications, pp. 1782-1789, Nov. 2005
[96] Hsiao-Hwa Chen, A. J. Han Vinck, Qi Bi, F. Adachi and L. B. Milstein, “Next
Generation CDMA Technologies,” IEEE Journal on Selected Areas in
Communications, pp. 1-3, ISSN 0733-8716, Jan 2006
[97] T. Kløve, “Classification of permutation codes of length 6 and minimum
distance 5,” Proceedings, ISITA, Honolulu, Hawaii, USA, pp. 465-468Nov. 5-8,
2000
[98] Janiszczak I., Staszewski R., “An improved bound for permutation arrays of
length 10. http://www.iem.uni-due.de/preprints/IJRS.pdf (downloaded 1st March
2011)
[99] P. P. Bergmans and T. M. Cover, “Cooperative broadcasting,” IEEE Trans-
actions on Information. Theory, vol. IT-20, pp. 317–324, May 1974

196
 Appendix

Z channel 23
check part 130, 174
Chien search 177
ciphertext 113

INDEX code
extended 31
KS 34
achievable rate 20 minimum distance 2
additive coding 148 repetition 173
additive white Gaussian noise 64 superimposed 34
Aloha 7 coding gain 70
array 9 collision 7
ARQ 3 concatenated code 88
attenuation 45 constrained coding 129
attenuation factor 46 control word 130
authentication 115, 120 cooperative coding 47
Automatic Repeat reQuest 3 coordinated access 21
correlator 78
background noise 81 cover 34
bandwidth 27, 77, 185 cryptosystem 113
efficiency 78, 84
Berlekamp Massey 176 d-constraint 132, 134
biometric 115, 120 decode-and-forward 47
broadcast channel 57, 186 defect
bus 46 matching 150
model 147
capacity -0 147
uncoordinated 20 -1 147
secrecy 97 compatible 150
channel 162 detect-and-forward 47
impulse noise 181 dimension division 21
Gaussian 185 distance
carrier frequency 138 minimum 2
channel spectrum 68
binary OR 20
binary XOR 20 efficiency 3
common 19 encoding matrix 2
Gaussian 185 energy 64, 65
M-ary OR 23 energy detector 32
M-ary Erasure 25 enrollment 115, 117
main 97 entropy 161
parallel 187, 188 conditional 162

197
 Appendix

joint 162 insertion 86

envelope detector 78, 80 iterative decoding 92
equivocation 97, 107, 162
erasure 2, 66 Kautz-Singleton 34
error key 113
correcting 2, 65 equation 176
detection 2 noisy 115
and erasure 66
locator 176 leakage 113, 120
Euclid 176 links in tandem 46

False Acceptance Rate (FAR) 118 majority vote 173

False Rejection Rate (FRR) 118 M-ary OR 23
FDMA 21 Maximum Likelihood 67
frequency-sharing 55 MFSK 27, 77
frequency selective fading 81 Middleton Class-A 178
minimal polynomial 165
Galois Field 164 minimum distance 2
Gaussian broadcast channel 57 modulated symbol 77
greatest common divisor 176 mother code 141
guessing probability 119 multi user tandem 53
multi user 19, 53
Hamming code 101 multiple access 20
hard decision 65 multiple access channel 19, 58
hard quantized 70 mutual information 162
hash 122
narrowband noise 80, 145
idle 21 noise
IDLP 110 mitigation 184
impulse entropy 116, 185
index A 178 noisy cipher 115
noise 80, 178 non-coherent detection 78, 138
strength 178
integrate-and-dump 138 ODP 140
intermediate receiver 47 optical pulse modulation 21
information optimum distance profile 140
rate 77
side 162 packet, 3
mutual 162 checksum 3
informed error rate 3
receiver 181 synchronization 3
transmitter 181 partly observed message 107

198
 Appendix

perfect secrecy 113 soft decision 63

permutation code 82 soft decoding 67
point-to-point 54 special element 35
polynomial squared Euclidean distance 67
minimal 165 storage capacity 148
irreducible 165 stuff bits 133
power allocation 53, 55 sub-slot 26
power line communication 45, 93 super channel 88
PPM 26 syndrome 101, 172
primitive element 164 former 171
product code 90 symbol
Pulse Position Modulation 26 avoidance 129
puncturing 140 look-ahead 133
duration 77, 133
Q-function 46, 65 systematic encoding 102

random signatures 27 tandem 46, 97

rate 20 TDMA 21
relay 49, 56 throughput 1, 20
repeater 47 time-sharing 55
repetition code 172 Titlebaum 31
RLL 132 transition 133
row transmission efficiency 20
extension 142 two access 58
deletion 140
RS-constrained code 90
Run Length Limited 132 un-coded 64
uncoordinated access 20
same-weight 144 undetected error probability 3
sampling instance 79 union of a set 34
secrecy capacity 97, 99 union bound 70
semi-systematic 104 user
separation 133 legal 118
side information 162 illegal 119
signatures
random 27 Vandermonde
constructed 31 determinant167
modulated 31 matrix 167
verification 115
single parity check code 71 Viterbi 76
sinusoidal waves 77
slot 26 water-filling 55, 187, 188

199
 Appendix

wiretap channel 95 write once memory 159

wiretapper 95

200
 Appendix

My coo-authorrs for thee referen

nced pap
pers

A.V. Kuuznetzov,
member oof IPPI, Mosccow, Russia Fangwei Fu
Guest IEM
M Nankeei University, China, Guestt IEM

S. Thokozani, An
nil Mengi,
PhD Univv. of Johannessburg, South PhD, IEM
Africa, vvisiting scientiist IEM

Yanlin
ng Chen
Yuan Luo,, PhD, IEM
Tjiao-Tonng University Shanghai
Guest ressearcher IEM,

Jacco Versveld
Guestt IEM, Univ. Witwatersrand
W d,
Chriistof Haslach, Southh Africa
M
PhD, IEM

Wim Hoeks
H
Universitty of Eindhoveen A. van
n Wijngaarden
Master Sttudent, 1983 PhD, IEM

201
 Appendix

Samwel Martirossian Jeroen Keunig,

Guest IEM
M, Univ. Duissburg-Essen Guestt IEM
Armenia Academy of Sciences
S Univeersity Eindhov
ven, The Neth
herlands

Vladim
mir Balakirsky
y, Yanlin
ng Chen,
Guest IEM
M, Russia PhD U
University Du
uisburg-Essen

Robert Schweikert,
Chai Mitrpant, PhD Audenns, former DL
LR Oberpfaffeenhofen,
Universitty Duisburg-E
Essen, 2004 Germ
many

Peter Gobeer Henndrik Ferreira

PhD, IEM
M, Univ. Duisbburg Essen Univ. of Johannesb
burg, South Affrica
Guestt IEM, Univ. Duisburg-Esse
D en

Young Gil
G Kim
Guest IEMM, Lutzz Lampe,
Seoul Unniversity, Soutth Korea Prof. UBC, Canadda, Humboldt Fellow,
Guestt IEM

Hiroyoshi Morita
M Francis Swart
Chofu Unniv. of Electroo-Comm, Japaan, Univ. of Johannesb
burg, South Affrica
Guest IEM
M, Univ. Duiisburg-Essen. Guesst IEM, Univ. Duisburg--Essen

202
 Appendix

Tadashi Wadayama
W
Jürgenn Häring
Guest IEM
M, Nagoya Unniversity, Japaan
2001, PhD, Univ. Duisburg
D Esseen

Martin
n Vinck, PhD D Univ.
Frans Willems,
W of Am
msterdam, the Netherlands
Univ. of E
Eindhoven, thhe Netherlandss

Vladimir Leevenshtein Victorr Papilaya, PhD

D
Keldish IInst., Moscow
w, RU, visitor IEM
I Studeent IEM, Univ
v. Duisburg-Esssen

Der Fen
ng Tseng
Wai Ho Moow, Nationnal Taiwan University
U of Science
Chinese U
University of HongKong Technology, Taipei
and T

Yunghsianng Sam Han

National Taiwan Univversity of Sciience Javad Yazdani, Un niversity
and Techhnology, Taipeei of Cenntral Lancash
hire, England

Niovi Paavlidou, Univeersity Bahram Honary,

of Thessaaloniki, Greecce Univeersity of Lancaster, England
d

203
 Appendix

Piet Schalkwijk Dr. Karel Post,

Universiity of Eindhovven Univeersity of Eindh
hoven

Theo
o Swarts
Hans Peter Foerster Univeersity of Johan
nnesburg
DLR Obeerpfaffenhofenn, Germany Southh Africa

Hsiao-HHwa Chen
Andre de
d Paepe
Nationnal Cheng Ku ung University
y
Universitty of Eindhovven
Tainaan City , 70101, Taiwan

204
 Appendix

To enable mathematiccians, computer experts andd telecommun nications engiineers to

engage inn uncomplicaated and transs-disciplinary collaboration n under one roof,
r the
Institute ffor Experimenntal Mathemaatics (IEM) waas founded, with
w the suppo ort of the
Volkswaggen Foundatioon, as a centrral scientific ffacility of thee former Univversity of
Essen in 1989. With the addition of o the Alfriedd Krupp von Bohlen und Halbach
Foundatioon Chair on 1 January 19 999, the Instiitute was exp panded in the area of
“Computter Networkinng Technolog gy”. The areeas of finite mathematics, digital
communiications, compputer network king technoloogy and theorry of numbers are all
representted at the IEM
M.

The prim mary objective of the Institu ute is to fosterr interactions between the fields of
mathemaatics, computeer science and a the enginneering scien nces. Severall of the
activities carried out byy scientists at IEM in pursuuit of this objeective are listed below:

- basic ressearch in alg gebra, theoryy of numberss, and algebrraic and

technical coding theory y;
- improvem ment of posssibilities for using comp puters in mathematic
b development algorithm
research by ms and more effficient softwaare;
- developmment of metho ods for digitaal communicaation data bacckup for
theoreticaal and practicaal applicationss.

The Workking Group onn Digital Com

mmunications focuses on prroblems in thee areas of
mmunication theory and daata security.
informatiion theory, com

205
Appendix

206