Download the full version of the ebook at

https://ebookultra.com

Error Control Coding From Theory to Practice

1st Edition Peter Sweeney

https://ebookultra.com/download/error-control-
coding-from-theory-to-practice-1st-edition-peter-
sweeney/

Explore and download more ebook at https://ebookultra.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Digital Speech Transmission Enhancement Coding and Error

Concealment 1st Edition Peter Vary

https://ebookultra.com/download/digital-speech-transmission-
enhancement-coding-and-error-concealment-1st-edition-peter-vary/

ebookultra.com

Non Binary Error Control Coding for Wireless Communication

and Data Storage 1st Edition Rolando Antonio Carrasco

https://ebookultra.com/download/non-binary-error-control-coding-for-
wireless-communication-and-data-storage-1st-edition-rolando-antonio-
carrasco/
ebookultra.com

Introduction to coding theory 2nd Edition Bierbrauer

https://ebookultra.com/download/introduction-to-coding-theory-2nd-
edition-bierbrauer/

ebookultra.com

Multicultural Education From Theory to Practice 1st

Edition Hasan Arslan

https://ebookultra.com/download/multicultural-education-from-theory-
to-practice-1st-edition-hasan-arslan/

ebookultra.com
Therapeutic Exercise From Theory to Practice 1st Edition
Michael Higgins

https://ebookultra.com/download/therapeutic-exercise-from-theory-to-
practice-1st-edition-michael-higgins/

ebookultra.com

Anthropology for Development From Theory to Practice 1st

Edition Robyn Eversole

https://ebookultra.com/download/anthropology-for-development-from-
theory-to-practice-1st-edition-robyn-eversole/

ebookultra.com

Multimedia Networking From Theory to Practice 1st Edition

Jenq-Neng Hwang

https://ebookultra.com/download/multimedia-networking-from-theory-to-
practice-1st-edition-jenq-neng-hwang/

ebookultra.com

Roadwork Theory and Practice 5th Edition Peter Kendrick

https://ebookultra.com/download/roadwork-theory-and-practice-5th-
edition-peter-kendrick/

ebookultra.com

Coding theory a first course Ling

https://ebookultra.com/download/coding-theory-a-first-course-ling/

ebookultra.com
Error Control Coding From Theory to Practice 1st
Edition Peter Sweeney Digital Instant Download
Author(s): Peter Sweeney
ISBN(s): 9780470852613, 0470852615
Edition: 1
File Details: PDF, 16.09 MB
Year: 2002
Language: english
ERROR CONTROL CODING
ERROR CONTROL CODING
From Theory to Practice

Peter Sweeney
University of Surrey, Guildford, UK

JOHN WILEY & SONS, LTD

Copyright © 2002 John Wiley & Sons, Ltd., Baffins Lane, Chichester,
West Sussex PO19 1UD, England

Phone (+44) 1243 779777

E-mail (for orders and customer service enquiries): cs- [email protected]

Visit our Home Page on www.wiley.co.uk or www.wiley.com

All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or
transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or
otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of
a licence issued by the Copyright Licensing Agency Ltd., 90 Tottenham Court Road, London W1P 0LP.
UK, without the permission in writing of the Publisher. Requests to the Publisher should be addressed
to the Permissions Department, John Wiley & Sons, Ltd., Baffins Lane, Chichester, West Sussex PO19
1UD, England, or emailed to [email protected], or faxed to (+44) 1243 770571.

Other Wiley Editorial Offices

John Wiley & Sons, Inc., 605 Third Avenue, New York, NY 10158–0012, USA

Jossey-Bass, 989 Market Street, San Francisco, CA 94103–1741, USA

Wiley-VCH Verlag GmbH, Pappelallee 3, D-69469 Weinheim, Germany

John Wiley & Sons Australia, Ltd., 33 Park Road, Milton, Queensland 4064, Australia

John Wiley & Sons (Asia) Pte Ltd., 2 Clementi Loop #02–01, Jin Xing Distripark, Singapore 129809

John Wiley & Sons Canada, Ltd., 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1

British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library

ISBN 0 470 84356 X

Typeset in 10/12pt Times by Kolam Information Services Pvt. Ltd, Pondicherry, India
Printed and bound in Great Britain by TJ International, Padstow, Cornwall
This book is printed on acid-free paper responsibly manufactured from sustainable forestry
in which at least two trees are planted for each one used for paper production.
1 The Principles of Coding in Digital Communications 1
1.1 Error Control Schemes 1
1.2 Elements of Digital Communication Systems 2
1.3 Source Encoding 2
1.4 Error Control Coding 3
1.5 Modulation 5
1.6 The Channel 7
1.7 Demodulation 8
1.7.1 Coherent demodulation 8
1.7.2 Differential demodulation 9
1.7.3 Soft-decision demodulation 10
1.8 Decoding 11
1.8.1 Encoding and decoding example 12
1.8.2 Soft-decision decoding 14
1.8.3 Alternative decoding approaches 15
1.9 Code Performance and Coding Gain 16
1.10 Information Theory Limits to Code Performance 18
1.11 Coding for Multilevel Modulations 21
1.12 Coding for Burst-Error Channels 22
1.13 Multistage Coding 24
1.14 Error Detection Based Methods 24
1.14.1 ARQ strategies 24
1.14.2 Error concealment 26
1.14.3 Error detection and correction capability of block
codes 26
1.15 Selection of Coding Scheme 27
1.15.1 General considerations 27
1.15.2 Data structure 28
1.15.3 Information type 29
1.15.4 Data rate 29
1.15.5 Real time data processing 30
1.15.6 Power and bandwidth constraints 30
vi CONTENTS

1.15.7 Channel error mechanisms 31

1.15.8 Cost 31
1.16 Conclusion 32
1.17 Exercises 33
1.18 References 34

2 Convolutional Codes 35
2.1 Introduction 35
2.2 General Properties of Convolutional Codes 35
2.3 Generator Polynomials 36
2.4 Terminology 37
2.5 Encoder State Diagram 38
2.6 Distance Structure of Convolutional Codes 39
2.7 Evaluating Distance and Weight Structures 39
2.8 Maximum Likelihood Decoding 41
2.9 Viterbi Algorithm 42
2.9.1 General principles 42
2.9.2 Example of viterbi decoding 42
2.9.3 Issues arising 45
2.10 Practical Implementation of Viterbi Decoding 45
2.11 Performance of Convolutional Codes 51
2.12 Good Convolutional Codes 53
2.13 Punctured Convolutional Codes 55
2.14 Applications of Convolutional Codes 56
2.15 Codes for Multilevel Modulations 57
2.16 Sequential Decoding 62
2.17 Conclusion 64
2.18 Exercises 64
2.19 References 66

3 Linear Block Codes 67

3.1 Introduction 67
3.2 Mathematics of Binary Codes 67
3.3 Parity Checks 68
3.4 Systematic Codes 69
3.5 Minimum Hamming Distance of a Linear Block Code 70
3.6 How to Encode - Generator Matrix 70
3.7 Encoding with the Parity Check Matrix 71
3.8 Decoding with the Parity Check Matrix 73
3.9 Decoding by Standard Array 75
3.10 Codec Design for Linear Block Codes 76
3.11 Modifications to Block Codes 78
3.12 Dorsch Algorithm Decoding 81
3.13 Conclusion 83
3.14 Exercises 83
3.15 References 85
 CONTENTS vii

4 Cyclic Codes 87
4.1 Introduction 87
4.2 Definition of a Cyclic Code 87
4.3 Example of a Cyclic Code 88
4.4 Polynomial Representation 88
4.5 Encoding by Convolution 89
4.6 Establishing the Cyclic Property 90
4.7 Deducing the Properties of a Cyclic Code 91
4.8 Primitive Polynomials 92
4.9 Systematic Encoding of Cyclic Codes 93
4.10 Syndrome of a Cyclic Code 94
4.11 Implementation of Encoding 94
4.12 Decoding 96
4.13 Decoder Operation 100
4.14 Multiple-Error Correction 100
4.15 Example of Multiple-Error Correction 101
4.16 Shortened Cyclic Codes 103
4.17 Expurgated Cyclic Codes 104
4.18 BCH Codes 106
4.19 Cyclic Codes for Burst-Error Correction 107
4.20 Conclusion 110
4.21 Exercises 110
4.22 References 112

5 Finite Field Arithmetic 113

5.1 Introduction 113
5.2 Definition of a Finite Field 113
5.3 Prime Size Finite Field GF(p) 114
5.4 Extensions to the Binary Field - Finite Field GF(2m) 115
5.5 Polynomial Representation of Finite Field Elements 117
5.6 Properties of Polynomials and Finite Field Elements 119
5.6.1 Roots of a polynomial 119
5.6.2 Minimum polynomial 120
5.6.3 Order of an element 120
5.6.4 Finite field elements as roots of a polynomial 120
5.6.5 Roots of an irreducible polynomial 120
5.6.6 Factorization of a polynomial 121
5.7 Fourier Transform over a Finite Field 121
5.8 Alternative Visualization of Finite Field Fourier Transform 123
5.9 Roots and Spectral Components 124
5.10 Fast Fourier Transforms 125
5.11 Hardware Multipliers using Polynomial Basis 127
5.12 Hardware Multiplication using Dual Basis 129
5.13 Hardware Multiplication using Normal Basis 131
5.14 Software Implementation of Finite Field Arithmetic 132
5.15 Conclusion 134
viii CONTENTS

5.16 Exercises 135

5.17 References 136

6 BCH Codes 137

6.1 Introduction 137
6.2 Specifying Cyclic Codes by Roots 137
6.3 Definition of a BCH Code 138
6.4 Construction of Binary BCH Codes 138
6.5 Roots and Parity Check Matrices 140
6.6 Algebraic Decoding 143
6.7 BCH Decoding and the BCH Bound 144
6.8 Decoding in the Frequency Domain 146
6.9 Decoding Examples for Binary BCH Codes 147
6.10 Polynomial Form of the Key Equation 149
6.11 Euclid's Method 149
6.12 Berlekamp—Massey Algorithm 151
6.13 Conclusion 152
6.14 Exercises 153
6.15 References 154

7 Reed Solomon Codes

7.1 Introduction
7.2 Generator Polynomial for a Reed Solomon Code
7.3 Time Domain Encoding for Reed Solomon Codes
7.4 Decoding Reed Solomon Codes
7.5 Reed Solomon Decoding Example
7.6 Frequency Domain Encoded Reed Solomon Codes
7.7 Further Examples of Reed Solomon Decoding
7.8 Erasure Decoding
7.9 Example of Erasure Decoding of Reed Solomon Codes
7.10 Generalized Minimum Distance Decoding
7.11 Welch—Berlekamp Algorithm
7.12 Singly Extended Reed Solomon Codes
7.13 Doubly Extended Reed Solomon Codes
7.14 Conclusion
7.15 Exercises
7.16 References

8 Performance Calculations for Block Codes

8.1 Introduction
8.2 Hamming Bound
8.3 Plotkin Bound
8.4 Griesmer Bound
8.5 Singleton Bound
8.6 Gilbert—Varsharmov Bound
8.7 Error Detection
 CONTENTS ix

8.8 Random-Error Detection Performance of Block Codes 181

8.9 Weight Distributions 182
8.10 Worst Case Undetected Error Rate 184
8.11 Burst-Error Detection 185
8.12 Examples of Error Detection Codes 185
8.13 Output Error Rates using Block Codes 186
8.14 Detected Uncorrectable Errors 188
8.15 Application Example - Optical Communications 190
8.16 Conclusion 192
8.17 Exercises 192

9 Multistage Coding 195

9.1 Introduction 195
9.2 Serial Concatenation 195
9.3 Serial Concatenation using Inner Block Code 196
9.3.1 Maximal length codes 196
9.3.2 Orthogonal codes 197
9.3.3 Reed Muller codes 198
9.3.4 High rate codes with soft-decision decoding 198
9.4 Serial Concatenation using Inner Convolutional Code 199
9.5 Product codes 200
9.6 Generalized Array Codes 203
9.7 Applications of Multistage Coding 206
9.8 Conclusion 208
9.9 Exercises 208
9.10 References 209

10 Iterative Decoding 211

10.1 Introduction 211
10.2 The BCJR Algorithm 211
10.3 BCJR Product Code Example 212
10.4 Use of Extrinsic Information 214
10.5 Recursive Systematic Convolutional Codes 215
10.6 MAP Decoding of RSC Codes 217
10.7 Interleaving and Trellis Termination 220
10.8 The Soft-Output Viterbi Algorithm 222
10.9 Gallager Codes 225
10.10 Serial Concatenation with Iterative Decoding 231
10.11 Performance and Complexity Issues 232
10.12 Application to Mobile Communications 233
10.13 Turbo Trellis-Coded Modulation 233
10.14 Conclusion 235
10.15 Exercises 235
10.16 References 236

Index 239
This page intentionally left blank
1
The principles of coding in digital
communications

1.1 ERROR CONTROL SCHEMES

Error control coding is concerned with methods of delivering information from a
source to a destination with a minimum of errors. As such it can be seen as a branch
of information theory and traces its origins to Shannon's work in the late 1940s.
The early theoretical work indicates what is possible and provides some insights
into the general principles of error control. On the other hand, the problems
involved in finding and implementing codes have meant that the practical effects
of employing coding are often somewhat different from what was originally
expected.
Shannon's work showed that any communication channel could be characterized
by a capacity at which information could be reliably transmitted. At any rate of
information transmission up to the channel capacity, it should be possible to transfer
information at error rates that can be reduced to any desired level. Error control can
be provided by introducing redundancy into transmissions. This means that more
symbols are included in the message than are strictly needed just to convey the
information, with the result that only certain patterns at the receiver correspond to
valid transmissions. Once an adequate degree of error control has been introduced,
the error rates can be made as low as required by extending the length of the code,
thus averaging the effects of noise over a longer period.
Experience has shown that to find good long codes with feasible decoding schemes
is more easily said than done. As a result, practical implementations may concentrate
on the improvements that can be obtained, compared with uncoded communica-
tions. Thus the use of coding may increase the operational range of a communication
system, reduce the error rates, reduce the transmitted power requirements or obtain a
blend of all these benefits.
Apart from the many codes that are available, there are several general techniques
for the control of errors, and the choice will depend on the nature of the data
and the user's requirements for error-free reception. The most complex techniques
fall into the category of forward error correction, where it is assumed that a code
capable of correcting any errors will be used. Alternatives are to detect errors and
request retransmission, which is known as retransmission error control, or to use
2 ERROR CONTROL CODING

inherent redundancy to process the erroneous data in a way that will make the errors
subjectively important, a method known as error concealment.
This chapter first looks at the components of a digital communication system.
Sections 1.3 to 1.8 then look in more detail at each of the components. Section 1.8
gives a simple example of a code that is used to show how error detection and
correction may in principle be achieved. Section 1.9 discusses the performance of
error correcting codes and Section 1.10 looks at the theoretical performance avail-
able. A number of more advanced topics are considered in Sections 1.11 to 1.14,
namely coding for bandwidth-limited conditions, coding for burst errors, multistage
coding (known as concatenation) and the alternatives to forward error correction.
Finally, Section 1.15 summarizes the various considerations in choosing a coding
scheme.

1.2 ELEMENTS OF DIGITAL COMMUNICATION

SYSTEMS
A typical communication system incorporating coding is shown in Figure 1.1. Error
control coding is applied after the source information is converted into digital format
by the source encoder. The separation between coding and modulation is conven-
tional, although it will be found later that there are instances where the two must be
designed together. At the receiver, the operations are carried out in reverse order
relative to the transmitter.
The functions are described in more detail in the following sections.

Error control Source

Modul encoder encoder „ -„ j
1 information
C
h
a
n
n
e
1

T
Demodu IaU"
Error control Source
received

decoder decoder

Figure 1.1 Coded communication system

1.3 SOURCE ENCODING

Information is given a digital representation, possibly in conjunction with techniques
for removal of any inherent redundancy within the data. The amount of information
 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 3

contained in any message is defined in terms of the probability p that the message is
selected for transmission. The information content H, measured in bits, is given by

H = log2(l/p)

For example, a message with a 1 % chance of being selected would contain

approximately 6.64 bits of information.
If there are M messages available for selection and the probability of message m is
denoted pm, the average amount of information transferred in a message is

subject to the constraint that Y^m=Q Pm = I-

If the messages are equiprobable, i.e. pm = 1/M, then the average information
transferred is just Iog2 (M). This is the same as the number of bits needed to represent
each of the messages in a fixed-length coding scheme. For example, with 256
messages an 8-bit code can be used to represent any messages and, if they are equally
likely to be transmitted, the information content of any message is also 8 bits.
If the messages are not equally likely to be transmitted, then the average infor-
mation content of a message will be less than Iog2 (M) bits. It is then desirable to find
a digital representation that uses fewer bits, preferably as close as possible to the
average information content. This may be done by using variable length codes such
as Huffman codes or arithmetic codes, where the length of the transmitted sequence
matches as closely as possible the information content of the message. Alternatively,
for subjective applications such as speech, images or video, lossy compression tech-
niques can be used to produce either fixed-length formats or variable-length formats.
The intention is to allow the receiver to reconstitute the transmitted information into
something that will not exactly match the source information, but will differ from it
in a way that is subjectively unimportant.

1.4 ERROR CONTROL CODING

Error control coding is in principle a collection of digital signal processing techniques
aiming to average the effects of channel noise over several transmitted signals. The
amount of noise suffered by a single transmitted symbol is much less predictable than
that experienced over a longer interval of time, so the noise margins built into the
code are proportionally smaller than those needed for uncoded symbols.
An important part of error control coding is the incorporation of redundancy into
the transmitted sequences. The number of bits transmitted as a result of the error
correcting code is therefore greater than that needed to represent the information.
Without this, the code would not even allow us to detect the presence of errors and
therefore would not have any error controlling properties. This means that, in theory,
any incomplete compression carried out by a source encoder could be regarded as
having error control capabilities. In practice, however, it will be better to compress the
4 ERROR CONTROL CODING

source information as completely as possible and then to re-introduce redundancy in a

way that can be used to best effect by the error correcting decoder.
The encoder is represented in Figure 1.2. The information is formed into frames to
be presented to the encoder, each frame consisting of a fixed number of symbols. In
most cases the symbols at the input of the encoder are bits; in a very few cases
symbols consisting of several bits are required by the encoder. The term symbol will
be used to maintain generality.
To produce its output, the encoder uses the symbols in the input frame and
possibly those in a number of previous frames. The output generally contains more
symbols than the input, i.e. redundancy has been added. A commonly used descrip-
tor of a code is the code rate (R) which is the ratio of input to output symbols in one
frame. A low code rate indicates a high degree of redundancy, which is likely to
provide more effective error control than a higher rate, at the expense of reducing the
information throughput.
If the encoder uses only the current frame to produce its output, then the code is
called a (n, k) block code, with the number of input symbols per frame designated k
and the corresponding number of output symbols n. If the encoder remembers a
number of previous frames and uses them in its algorithm, then the code is called a
tree code and is usually a member of a subset known as convolutional codes. In this
case the number of symbols in the input frame will be designated k0 with n0 symbols
in the output frame. The encoder effectively performs a sliding window across the
data moving in small increments that leave many of the same symbols still within the
encoder window, as shown in Figure 1.3. The total length of the window, known as

0
f 1
s 2

Output
buffer
k-\
n-\

Figure 1.2 Encoder

input
v-bit encoder memory frame

AT-bit encoder constraint length

Figure 1.3 Sliding window for tree encoder

 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 5

the input constraint length (K), consists of the input frame of k0 symbols plus the
number of symbols in the memory. This latter parameter is known as memory
constraint length (v).
In more complex systems the encoding may consist of more than one stage and
may incorporate both block and convolutional codes and, possibly, a technique
known as interleaving. Such systems will be considered in later sections.
One property that will be shared by all the codes in this book is linearity. If we
consider a linear system we normally think in terms of output being proportional to
input (scaling property). For a linear system we can also identify on the output the
sum of the separate components deriving from the sum of two different signals at the
input (superposition property). More formally, if the system performs a function/on
an input to produce its output, thenf(cx)= c x f(x) (scaling)

f(x + y) =f(x) +f(y) (superposition)

where c is a scalar quantity, x and y are vectors.

Now the definition of a linear code is less restrictive than this, in that it does not
consider the mapping from input to output, but merely the possible outputs from the
encoder. In practice, however, a linear system will be used to generate the code and so
the previous definition will apply in all real-life cases.
The standard definition of a linear code is as follows:
• Multiplying a code sequence by a valid scalar quantity produces a code sequence.
• Adding two code sequences produces a code sequence.
The general rules to be followed for multiplication and addition are covered in
Chapter 5 but for binary codes, where the only valid scalars are 0 and 1, multipli-
cation of a value by zero always produces zero and multiplication by 1 leaves the
value unchanged. Addition is carried out as a modulo-2 operation, i.e. by an exclu-
sive-OR function on the values.
A simple example of a linear code will be given in Section 1.8. Although the
definition of a linear code is less restrictive than that of a linear system, in practice
linear codes will always be produced by linear systems. Linear codes must contain the
all-zero sequence, because multiplying any code sequence by zero will produce an all-
zero result.

1.5 MODULATION
The modulator can be thought of as a kind of digital to analogue converter, preparing
the digital code-stream for the real, analogue world. Initially the digital stream is put
into a baseband representation, i.e. one in which the signal changes at a rate compar-
able with the rate of the digital symbols being represented. A convenient representa-
tion is the Non Return to Zero (NRZ) format, which represents bits by signal levels of
+ V or — V depending on the bit value. This is represented in Figure 1.4.
 ERROR CONTROL CODING

+K,
time

1 2 3 4 5 6 7 8 91 0

Figure 1.4 Binary NRZ stream

Although it would be possible to transmit this signal, it is usual to translate it into a

higher frequency range. The reasons for this include the possibility of using different
parts of the spectrum for different transmissions and the fact that higher frequencies
have smaller wavelengths and need smaller antennas. For most of this text, it will be
assumed that the modulation is produced by multiplying the NRZ baseband signal by
a sinusoidal carrier whose frequency is chosen to be some multiple of the transmitted
bit rate (so that a whole number of carrier cycles are contained in a single-bit interval).
As a result, the signal transmitted over a single-bit interval is either the sinusoidal
carrier or its inverse. This scheme is known as Binary Phase Shift Keying (BPSK).
It is possible to use a second carrier at 90° to the original one, modulate it and add
the resulting signal to the first. In other words, if the BPSK signal is ± cos (2nfct),
where fc is the carrier frequency and t represents time, the second signal is
± sin (2nfci) and the resultant is

s(t) = V2cos(2nfct + in/4) / = -3, - 1, + 1, + 3

This is known as Quadriphase Shift Keying (QPSK) and has the advantage over
BPSK that twice as many bits can be transmitted in the same time and the same
bandwidth, with no loss of resistance to noise. The actual bandwidth occupied by the
modulation depends on implementation details, but is commonly taken to be 1 Hz
for one bit per second transmission rate using BPSK or 0.5 Hz using QPSK.
A phase diagram of QPSK is shown in Figure 1.5. The mapping of the bit values
onto the phases assumes that each of the carriers is independently modulated using
alternate bits from the coded data stream. It can be seen that adjacent points in the
diagram differ by only one bit because the phase of only one of the two carriers has
changed. A mapping that ensures that adjacent points differ by only one bit is known
as Gray Coding.

sin(2nfct)
01 11

cos(2nfct)

00 10

Figure 1.5 Gray-coded QPSK phase diagram

 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 7

Other possible modulations include Frequency Shift Keying (FSK), in which the
data determines the frequency of the transmitted signal. The advantage of FSK is
simplicity of implementation, although the resistance to noise is less than BPSK or
QPSK. There are also various modulations of a type known as Continuous Phase
Modulation, which minimize phase discontinuities between transmitted waveforms to
improve the spectral characteristics produced by nonlinear power devices.
In bandwidth-limited conditions, multilevel modulations may be used to achieve
higher bit rates within the same bandwidth as BPSK or QPSK. In M-ary Phase Shift
Keying (MPSK) a larger number of transmitted phases is possible. In Quadrature
Amplitude Modulation (QAM) a pair of carriers in phase quadrature are each given
different possible amplitudes before being added to produce a transmitted signal with
different amplitudes as well as phases. QAM has more noise resistance than equiva-
lent MPSK, but the variations in amplitude may cause problems for systems involv-
ing nonlinear power devices. Both QAM and MPSK require special approaches to
coding which consider the code and the modulation together.

1.6 THE CHANNEL

The transmission medium introduces a number of effects such as attenuation, distor-
tion, interference and noise, making it uncertain whether the information will be
received correctly. Although it is easiest to think in terms of the channel as introdu-
cing errors, it should be realized that it is the effects of the channel on the demodu-
lator that produce the errors.
The way in which the transmitted symbols are corrupted may be described using
the following terms:

• Memoryless channel - the probability of error is independent from one symbol to

the next.

• Symmetric channel - the probability of a transmitted symbol value i being received

as a value j is the same as that of a transmitted symbol value j being received as
i, for all values of i and j. A commonly encountered example is the binary
symmetric channel (BSC) with a probability p of bit error, as illustrated in
Figure 1.6.

1-p

Figure 1.6 Binary symmetric channel

8 ERROR CONTROL CODING

• Additive White Gaussian Noise (AWGN) channel - a memoryless channel in

which the transmitted signal suffers the addition of wide-band noise whose ampli-
tude is a normally (Gaussian) distributed random variable.

• Bursty channel - the errors are characterized by periods of relatively high symbol
error rate separated by periods of relatively low, or zero, error rate.

• Compound (or diffuse) channel - the errors consist of a mixture of bursts and
random errors. In reality all channels exhibit some form of compound behaviour.

Many codes work best if errors are random and so the transmitter and receiver
may include additional elements, an interleaver before the modulator and a deinter-
leaver after the demodulator to randomize the effects of channel errors. This will be
discussed in Section 1.12.

1.7 DEMODULATION

1.7.1 Coherent demodulation

The demodulator attempts to decide on the values of the symbols that were trans-
mitted and pass those decisions on to the next stage. This is usually carried out by
some sort of correlation with replicas of possible transmitted signals. Consider, for
example, the case of BPSK. The correlation is with a single fixed phase of the carrier,
producing either a positive or a negative output from the detector. In the absence of
noise, the detected signal level can be taken as ±\/E~r where Er is the energy in each
received bit. The effect of an AWGN channel will be to add a noise level n sampled
from a Gaussian distribution of zero mean and standard deviation a. The probability
density is given by

Gaussian noise has a flat spectrum and the noise level is often described by its
Single-sided Noise Power Spectral Density, which is written N0. The variance, <r2, of
the Gaussian noise, integrated over a single-bit interval, will be No/2. In fact it can be
considered that there is a total noise variance of N0 with half of this acting either in
phase or in antiphase to the replica and the other half in phase quadrature, therefore
not affecting the detector. The performance of Gray-coded QPSK is therefore exactly
the same as BPSK because the two carriers can be demodulated as independent
BPSK signals, each affected by independent Gaussian noise values with the same
standard deviation.
The demodulator will make its decision based on the sign of the detected signal.
If the received level is positive, it will assume that the value corresponding to the
replica was transmitted. If the correlation was negative, it will assume that the other
value was transmitted. An error will occur, therefore, if the noise-free level is -\fEr
 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 9

and a noise value greater than +\fEr is added, or if the noise-free level is +\fE~r and a
noise value less than —-/fir is added. Considering only the former case, we see that
the probability of error is just the probability that the Gaussian noise has a value
greater than +^/Er•.
cc

p = -4= f e-"2^° dn

Substituting t — n/^JN® gives

oc
,
where —
7 f
v^J

The function erfc(x) is known as the complementary error function and its values
are widely available in tabulated form. Note that the maximum value of erfc(x) is 1 .0,
so that the maximum bit error probability is 0.5. This makes sense because we could
guess bit values with 50% probability without attempting to receive them at all.

1.7.2 Differential demodulation

One further complication is commonly encountered with BPSK or QPSK transmis-

sions. In the absence of other information, it is impossible for the receiver to
determine absolute phase so as to know which of the two phases represents the
value 0 and which represents 1. This is because delay in the transmission path, which
is equivalent to phase shift, will not be known. The representation of bit values is
therefore often based on the difference between phases. Depending on the precise
demodulation method, this is known either as Differentially Encoded Phase Shift
Keying (DEPSK) or as Differential Phase Shift Keying (DPSK). The two are identical
from the modulation point of view, with the bit value 0 normally resulting in a
change of phase and the bit value 1 resulting in the phase remaining the same. The
receiver may not know absolute phase values, but should be able to tell whether the
phase has changed or remained the same. The differences in the demodulator
implementation may be summarized as follows:

• DEPSK – The demodulator maintains a replica of one of the two carrier phases
and correlates the received signal with this replica as for normal PSK. It then
compares the sign of the correlation with the previous correlation value; a change
of sign indicates data bit 0 and the same sign indicates data bit 1. Compared with
PSK, there will now be a bit error either when the phase is received wrongly and the
10 ERROR CONTROL CODING

previous phase was correct or when the phase is received correctly and the previous
phase was wrong. Thus noise that would cause a single-bit error in a BPSK
demodulator will cause two consecutive bit errors in the DEPSK demodulator
and the bit error probability is approximately twice the above BPSK expression.

• DPSK - The demodulator uses the previously received phase as the replica for the
next bit. Positive correlation indicates data value 1, negative correlation indicates
data value 0. The bit errors again tend to correlate in pairs, but the overall
performance is worse. In fact the bit error probability of DPSK follows a different
shape of curve:

1.7.3 Soft-decision demodulation

In some cases the demodulator's decision will be easy; in other cases it will be
difficult. In principle if errors are to be corrected it is better for the demodulator to
pass on the information about the certainty of its decisions because this might assist
the decoder in pinpointing the positions of the likely errors; this is called soft-decision
demodulation. We could think of it as passing on the actual detected level as a real
number, although in practice it is likely to have some sort of quantization. Eight-level
quantization is found to represent a good compromise between complexity and
performance.
Since the purpose of soft decisions is to assist decoding, it is useful to relate the
demodulator output to probabilistic measures of performance. One commonly
adopted measure is known as the log-likelihood ratio, defined as log[/Kl|r/)//?(0|r,)].
This metric is required in an important decoding method to be described in Chapter
10 and can be used for other decoding methods too. The computation of the value
may appear difficult, however we note that

Assuming that values 0 and 1 are equiprobable, log[p(l)/p(Q)] = 0 and so the

assigned bit value for received level r, is equal to log[/?(ri|l)//Kr/|0)]- This value can
be calculated given knowledge of the signal level and the noise statistics. Note that it
ranges from — oo (certain 0) to +00 (certain 1).
Assuming that we have Gaussian noise, the probability density function at a
received value ri from a noise-free received value x is

The appropriate values of x for bit values 1 and 0 are +\TEr and —\fEr- Thus the
2 2
log-likelihood ratio i s proportional t o
 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 11

Now

^.,-(n-\
log
#o No

Hence we find that

r<r,-|0)
V /[ * / I "IV ">
4r
I

In other words, the log-likelihood ratio is linear with the detected signal level and is
equal to the channel E r /N 0 , multiplied by four times the detected signal (normalized
to make the noise-free levels equal to -f/ — 1).
Note that the mapping adopted here from code bit values to detected demodulator
levels is opposite to that conventionally used in other texts. The conventional
mapping is that bit value 0 maps onto +1 and bit value 1 onto — 1 . The advantage
is that the exclusive-OR operation in the digital domain maps onto multiplication in
the analog domain. The disadvantage is the potential confusion between bit value
1 and analog value +1.
Because of the linearity of the log-likelihood ratio, the quantization boundaries of
the demodulator can be set in roughly linear steps. The question remains, however, as
to what size those steps should be. It can be shown that, for Q-level quantization, the
optimum solution is one that minimizes the value of

where p(jlc) represents the probability of a received value j given that symbol c was
transmitted. Massey [1] described an iterative method of finding the optimum solution
with nonuniform arrangement of boundaries, but the above value can easily be
calculated for different linear spacings to find an approximate optimum. For example,
with E r / N 0 around 2 dB, it is found that uniformly spaced quantization boundaries are
close to optimum if the spacing is 1/3, i.e. the boundaries are placed at —1, — 2/3
-1/3,0, +1/3, + 2/3, + 1. The use of such a scheme will be described in Section 1.8.2.

1.8 DECODING
The job of the decoder is to decide what the transmitted information was. It has the
possibility of doing this because only certain transmitted sequences, known as code-
words, are possible and any errors are likely to result in reception of a non-code
sequence. On a memoryless channel, the best strategy for the decoder is to compare
the received sequence with all the codewords, taking into account the confidence in
the received symbols, and select the codeword which is closest to the received
sequence as discussed above. This is known as maximum likelihood decoding.
12 ERROR CONTROL CODING

1.8.1 Encoding and decoding example

Consider, for example, the block code shown in Table 1.1. This code is said to be
systematic, meaning that the codeword contains the information bits and some other
bits known as parity checks because they have been calculated in some way from the
information. It can be seen that any codeword differs in at least three places from any
other codeword. This value is called the minimum Hamming distance or, more briefly,
minimum distance of the code. Consequently, if a single bit is wrong, the received
sequence is still closer to the transmitted codeword, but if two or more bits are
wrong, then the received sequence may be closer to one of the other codewords.
This code is linear and for any linear code it is found that the distance structure is
the same from any codeword. For this example, starting from any codeword there are
two sequences at a distance of 3 and one at a distance of 4. Thus the code properties
and the error-correction properties are independent of the sequence transmitted. As a
consequence, the minimum distance of the code can be found by comparing each
nonzero sequence with the all-zero sequence, finding the nonzero codeword with the
smallest number nonzero symbols. The count of nonzero symbols is known as the
weight of the sequence and the minimum weight of the code is equal to the minimum
distance.
Let us now assume that information 10 has been selected and that the sequence
10101 is therefore transmitted. Let us also assume that the received bits are hard-
decision quantized. If the sequence is received without error, it is easy to identify it in
the table and to decode. If there are errors, however, things will be more difficult and
we need to measure the number of differences between the received sequence and
each codeword. The measure of difference between sequences is known as Hamming
distance, or simply as distance between the sequences. Consider first the received
sequence 00101. The distance to each codeword is shown in Table 1.2.
In this case we can see that we have a clear winner. The transmitted sequence has
been selected as the most likely and the decoding is correct.

Table 1.1 Example block code

Information Codeword
00 00000
01 01011
10 10101
11 11110

Table 1.2 Distances for sequence 00101

Codeword Distance
00000 2
01011 3
10101 1
11110 4
 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 13

The previous example had an error in an information bit, but the result will be the
same if a parity check bit is wrong. Consider the received sequence 10111. The
distances are shown in Table 1.3. Again the sequence 10101 is chosen. Further
examples are left to the reader, but it will be found that any single-bit error can be
recovered, regardless of the position or the codeword transmitted.
Now let us consider what happens if there are two errors. It will be found that there
are two possibilities.
Firstly, consider the received sequence 11111. The distances are shown in Table 1.4.
In this case, the codeword 11110 is chosen, which is wrong. Moreover, the decoder has
decided that the final bit was wrong when in fact it was correct. Because there are at
least three differences between any pair of codewords, the decoder has made an extra
error on top of the two made by the channel, in effect making things worse.
Finally, consider the received sequence 11001, whose distances to the codewords
are shown in Table 1.5. In this case, there are two problems in reaching a decision.
The first, and obvious, problem is that there is no clear winner and, in the absence of
other information, it would be necessary to choose randomly between the two most
likely codewords. Secondly, we predicted at the outset that only single errors would
be correctable and the decoder may have been designed in such a way that it refuses
to decode if there is no codeword within a distance 1 of the received sequence. The
likely outcome for this example, therefore, is that the decoder will be unable to

Table 1.3 Distances for sequence 10111

Codeword Distance

00000 4
01011 3
10101 1
11110 2

Table 1.4 Distances for sequence 11111

Codeword Distance

00000 5
01011 2
10101 2
11110 1

Table 1.5 Distances for sequence 11001

Codeword Distance
00000 3
01011 2
10101 2
11110 3
14 ERROR CONTROL CODING

choose the most likely transmitted codeword and will indicate to the user the
presence of detected uncorrectable errors. This is an important outcome that may
occur frequently with block codes.

1.8.2 Soft-decision decoding

The probability that a sequence c of length n was transmitted, given the received
sequence r, is OJ'rJXc/lr,-). We wish to maximize this value over all possible code
sequences. Alternatively, and more conveniently, we take logarithms and find the
maximum of J]/=o l°§[/Kc/lr/)]- This can be carried out by a correlation process,
which is a symbol-by-symbol multiplication and accumulation, regarding the code
bits as having values +1 or — 1. Therefore we would be multiplying the assigned
probability by 1 for a code bit of 1 and by -1 for a code bit of 0. For hard decisions, a
codeword of length n at a distance d from the received sequence would agree in n — d
places and disagree in d places with the received sequence, giving a correlation metric
of 2n— d. Obviously choosing the codeword to maximize this metric would yield the
same decoding result as the minimum distance approach.
Even with soft decisions, we can adopt a minimum distance view of decoding and
minimize 53/=o {1 ~ 1°8 [/>(c/lr»)] }• The correlation and minimum distance approaches
are again identical provided we have an appropriate measure of distance. If the
received bits are given values vi equal to log[/?(l|r/)], then the distance to a bit value
1 is 1 — v/, the distance to a bit value 0 is v, and we maximize probability by minimizing
this measure of distance over all codewords.
The maximization of probability can also be achieved by maximizing some other
function that increases monotonically with it. This is the case for the log-likelihood
ratio log [p(\ |r/)/XO|r,-)]. To decode, we can maximize £], c, log[p(r,11 )//?(r,| 1)] where
Ci is taken as having values ± 1. This again corresponds to carrying out a correlation
of received log-likelihood ratios with code sequences.
As discussed in Section 1.7.3, it is likely that the received levels will be quantized.
For 8-level quantization, it might be convenient to use some uniform set of metric
values depending on the range within which the detected bit falls. Such a scheme is
shown in Figure 1.7.
Bearing in mind the fact that the log-likelihood ratio is linear with the analog
detected level from the demodulator, then the only deviation from an ideal 8-level

quantized
metric value

-3.5 -2,,-,, -0.5 +0.5 + 1.5 +2.5 +3.5

-1 0
normalized
"* detected value

Figure 1.7 Quantization boundaries for soft-decision demodulation

 15

quantization is that the end categories (000 and 111) extend to — oo and +00 and
therefore should have larger metrics associated with them. The effect on perform-
ance, however, is negligible. For E r / N 0 = 2 dB, the optimum soft-decision metric
values associated with this quantization arrangement are —3.85, —2.5, —1.5, —0.5,
+0.5, +1.5, +2.5, +3.85. Therefore the proposed metrics of —3.5 to +3.5 are very
close to optimum.
The assigned values can be scaled and offset in any convenient manner, so the
scheme in Figure 1.7 is equivalent to having bit values of (—7, —5, —3, —1, +1, +3,
+5, +7) or (0, 1, 2, 3, 4, 5, 6, 7). This last form is convenient for implementation of a
3-bit interface to the decoder.
Applying the correlation approach to a soft-decision case, the example in Table 1.4
might become a received sequence +2.5 +0.5 +1.5 +0.5 +3.5 with correlation values
as shown in Table 1.6.
The maximum correlation value indicates the decoder decision. In this case, the
decoder selects the correct codeword, illustrating the value of soft decisions from the
demodulator.

Table 1.6 Correlations for soft-decision sequence

+2.5+0.5+1.5+0.5+3.5
Codeword Correlation
-1 -1 -1 -1 -1 -8.5
-1+1-1+1+1 +0.5
+ 1-1+1-1+1 +6.5
+ 1 +1 +1 +1 -1 +1.5

1.8.3 Alternative decoding approaches

Although conceptually very simple, the method described above is very complex to
implement for many realistic codes where there may be very many codewords. As a
result, other decoding methods will need to be studied. For example, the parity
checks for the above code were produced according to very simple rules. Numbering
the bits from left to right as bits 4 down to 0, bits 4 and 3 constitute the information
and the parity bits are

bit 2 == bit 4
bit 1 = bit 3
bit 0 = bit 4 0 bit 3

The symbol © denotes modulo-2 addition or exclusive-OR operation. Considering

only hard decisions, when a sequence is received, we can simply check whether the
parity rules are satisfied and we can easily work out the implications of different error
patterns. If there are no errors, all the parity checks will be correct. If there is a single-
bit error affecting one of the parity bits, only that parity check will fail. If bit 4 is in
16 ERROR CONTROL CODING

error, parity bits 2 and 0 will be wrong. If bit 3 is in error, parity bits 1 and 0 will be
wrong. If both parity bits 2 and 1 fail, the error is uncorrectable, regardless of
whether parity bit 0 passes or fails.
We can now construct some digital logic to check the parity bits and apply the
above rules to correct any correctable errors. It will be seen that applying the rules
will lead to the same decodings as before for the examples shown. In the final
example case, where the sequence 11001 was received, all three parity checks fail.
This type of decoding procedure resembles the methods applied for error correc-
tion to many block codes. Note, however, that it is not obvious how such methods
can incorporate soft decisions from the demodulator. Convolutional codes, however,
are decoded in a way that is essentially the same as the maximum likelihood method
and soft decisions can be used.

1.9 CODE PERFORMANCE AND CODING GAIN

We saw earlier that we can obtain a theoretical expression for the bit error probabil-
ity of BPSK or QPSK on the AWGN channel in terms of the ratio of energy per
received bit to single-sided noise power spectral density, E r /N 0 . It is convenient to do
the same for systems that employ coding, however we first have to solve a problem of
comparability. Coding introduces extra bits and therefore we have to increase either
the time to send a given message or else the bandwidth (by transmitting faster). Either
case will increase the total noise in the message; in the first case because we get noise
from the channel for a longer time, in the second case because more noise falls within
the bandwidth.
The answer to this problem is to assess the error performance of the link in terms of
Eb/No, the ratio of energy per bit of information to noise power spectral density. Thus
when coding is added, the number of bits of information is less than the number of
transmitted bits, resulting in an increase in E b /N 0 relative to E r /N 0 . For example, if
100 bits of information are to be sent using a rate 1/2 code, 200 bits must be
transmitted. Assuming that we maintain the transmitted bit rate and power, the energy
in the message is doubled, but the amount of information remains the same. Energy
per bit of information is therefore doubled, an increase of 3 dB. This increase acts as a
penalty that the code must overcome if it is to provide real gains. The performance
curve is built up in three stages as explained below.
As the first stage, the curve of bit error rate (BER) against E b /N 0 (the same as
Er/N0 in this case) is plotted for the modulation used. The value of Eb/No is usually
measured in dB and the bit error rate is plotted as a logarithmic scale, normally
covering several decades, e.g. from 10 - ' to 10-6. The second stage is the addition of
coding without consideration of the changes to bit error rates. For a fixed number
of transmitted bits, the number of information bits is reduced, thus increasing the
value of E b /N 0 relative to Er/N0 by a factor 1 / R, or by 10 log, 0 (l//?)dB. The third
stage is to consider the effect of coding on bit error rates; this may be obtained either
by simulation or by calculation. For every point on the uncoded performance
curve, there will therefore be a corresponding point a fixed distance to the right of
 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 17

it on the coded performance curve showing a different, in many cases lower, bit error
rate.
An example is shown in Figure 1.8 which shows the theoretical performance of
a BPSK (or QPSK) channel, uncoded and with a popular rate 1/2 convolutional
code. The code performance is plotted both with hard-decision demodulation and
with unquantized soft decisions, i.e. real number output of detected level from the
demodulator.
It can be seen that without coding, the value of E b / N 0 needed to achieve a bit error
rate of 10 -5 is around 9.6 dB. This error rate can be achieved with coding at
E b /N 0 around 7.1dB using hard-decision demodulation or around 4.2 dB using
unquantized soft-decision demodulation. This is expressed by saying that the coding
gain at a BER of 10 -5 is 2.5dB (hard-decision) or 5.4 dB (soft-decision). Real life
decoding gains would not be quite so large. The use of 8-level, or 3-bit, quantization
of the soft decisions reduces the gain by around 0.25 dB. There may also be other
implementation issues that affect performance. Nevertheless, gains of 4.5 to 5dB can
be expected with this code.
The quoted coding gain must be attached to a desired bit error rate, which in turn
will depend on the application. Note that good coding gains are available only for
relatively low required bit error rates and that at higher error rates the gain may be
negative (i.e. a loss). Note also that the quoted bit error rate is the error rate coming
out of the decoder, not the error rate coming out of the demodulator. In the soft-
decision example, the demodulator is working at E r /N 0 around 1.2dB, producing a
BER of around 5 x !0-2 out of the demodulator.
If we know the minimum distance of a block code, or the value of an equivalent
parameter called free distance for a convolutional code, we can find the asymptotic

2 3 4 5 6 7 8 9 1 0
l.0E+00

l.0E-01

Uncoded |
- Hard-decision!
Soft-decision !

Eb/N0 (dB)

Figure 1.8 Performance of rate 1/2 convolutional code

18 ERROR CONTROL CODING

coding gain, i.e. the gain that would be delivered if vanishingly small decoded error
rates were required. For unquantized soft-decision decoding of a rate R code with
distance d between the closest code sequences, the asymptotic gain is

(1.2)

If we have only hard decisions from the demodulator and can correct up to / errors
then
G
asymptotic = 10 log,0 [R(t + 1)] (1.3)

From the earlier simple block code example, we can see that the expected value of t
would be the largest value that is less than half the value of d. Thus the value of d in
Equation (1.2) is just less than twice the value of / + 1 in Equation (1.3). Asymptotic
coding gains are therefore almost 3 dB higher when unquantized soft-decision decod-
ing is used. As stated above, the use of 8-level quantization reduces the gain by about
0.25 dB.
Although we have solved one issue of comparability by the use of Eb/N0, there is
another that is regularly ignored. If we look at an uncoded channel and a coded
channel with the same BER, the characteristics will be completely different. On the
AWGN channel, the errors will occur at random intervals. On the coded channel
there will be extended error-free intervals interspersed with relatively dense bursts
of errors when the decoder fails. Thus if we are interested in error rates on larger
units of transmission, frames, packets or messages, the coded channel at the same
BER will give fewer failures but more bit errors in corrupted sections of transmission.
Assessing coding gain by comparing coded and uncoded channels with the same
BER may therefore be unfair to the coded channel. For example, out of 100 messages
sent, an uncoded channel might result in 10 message errors with one bit wrong in
each. A coded channel might produce only one message error but 10 bit errors within
that message. The bit error rates are the same, but the message error rate is better
on the coded channel. Add to this the fact that the detection of uncorrectable
errors is rarely taken into account in a satisfactory way (a significant issue for
many block codes), coding regularly delivers benefits that exceed the theoretical
figures.

1.10 INFORMATION THEORY LIMITS TO CODE

PERFORMANCE
We have now seen the sort of benefits that coding provides in present day practice
and the ways to find asymptotic coding gain based on knowledge of simple code
parameters. As yet we have not seen how to do detailed error rate calculations as
these require a more detailed knowledge of code structure. Nevertheless, it is worth
making a comparison with the results obtained from Shannon's work on information
theory to show that, in some respects, coded systems have still some way to go.
 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 19

Shannon showed that, using an average of all possible codes of length n, the error
rate over the channel is characterized by a probability of message error

Pe<e-'lE(Rl} (1.4)

where E, which is a function of the information rate, is called the random coding
error exponent. Any specific code will have its own error exponent and the greater
the error exponent the better the code, but there are calculable upper and lower
bounds to the achievable value of E. In particular, a positive error exponent is
achievable provided RI is less than some calculable value called the channel capacity.
Provided a positive error exponent can be obtained, the way to achieve lower error
probabilities is to increase the length of the code.
As was seen in Section 1.9, codes have a calculable asymptotic coding gain and
thus at high signal-to-noise values the error rates reduce exponentially with E b /N 0 , as
in the uncoded case. The error exponent is therefore proportional to Eb/N0.. The
difficulty with known codes is maintaining the error exponent while the length is
increased. All known codes produced by a single stage of encoding can hold
their value of error exponent only by reducing the rate to zero as the code length
increases towards infinity. For example, an orthogonal signal set, which can be
achieved by Frequency Shift Keying or by means of a block code, is sometimes
quoted as approaching the theoretical capacity on an AWGN channel as the signal
set is expanded to infinity. Unfortunately the bandwidth efficiency or the code rate
reduces exponentially at the same time. This limitation can be overcome by the use of
multistage encoding, known as concatenation, although even then the error expo-
nents are less than the theoretically attainable value. Nevertheless, concatenation
represents the closest practicable approach to the predictions of information theory,
and as such is a technique of increasing importance. It is treated in more detail in
Chapters 9 and 10.
As the most widely available performance figures for error correcting codes are for
the additive white Gaussian noise (AWGN) channel, it is interesting to look at the
theoretical capacity of such a channel. The channel rate is given by the Shannon-
Hartley theorem:

(1.5)

where B is bandwidth, S is signal power and N is noise power within the bandwidth.
This result behaves roughly as one might expect, the channel capacity increasing with
increased bandwidth and signal-to-noise ratio. It is interesting to note, however, that
in the absence of noise the channel capacity is not bandwidth-limited. Any two
signals of finite duration are bound to show differences falling within the system
bandwidth, and in the absence of noise those differences will be detectable.
Let N = B-No and S = R}Eh (NQ is the single-sided noise power spectral density,
RI is rate of information transmission (< C) and Eh is energy per bit of information),
then
20 ERROR CONTROL CODING

In the limit of infinite bandwidth, using the fact that Iog2 (jc) = loge (jc)/ log,, 2
gives

As bandwidth approaches infinity, the channel capacity is given by

For transmission at the channel capacity, (R1 = C):

(1.6)

This means that we should be able to achieve reliable communications at the

channel capacity with values of E b /N 0 as low as -1.6dB. The channel capacity is
however proportional to the information rate; increasing the rate for a fixed value of
Eb/N 0 increases the signal power and therefore the channel capacity. Thus at - 1 .6 dB
we should be able to achieve reliable communications at any rate over an AWGN
channel, provided we are willing to accept infinite bandwidth.
If instead we constrain the bandwidth and set R1 = r\B, where r\ is bandwidth
efficiency of the modulation/coding scheme, then

For transmission at the channel capacity (R\ — C), therefore

«V-1) (1.7)
Wo 1
This value can be thought of as imposing an upper limit to the coding gain
achievable by a particular coding and modulation scheme. The value of Eb/N0 to
deliver the desired error rate on the uncoded channel can be determined from the
modulation performance, and the corresponding coded value must be at least that
given by equation (1.7). In practice, these coding gains are difficult to achieve.
If we were to use a rate 1/2 code on a QPSK channel, a fairly common arrange-
ment, the value of rj is around 1.0, giving E b /N 0 = 1 ( = OdB). As has been seen
earlier, a rate 1/2 convolutional code may need over 4.5 dB to deliver a BER of 10-5.
It therefore falls well short of the theoretical maximum gain.
It must be stressed that Shannon merely proved that it was possible by coding to
obtain reliable communications at this rate. There is no benefit, however, in having a
 21

good code if one does not know how to decode it. Practical codes are designed with a
feasible decoding method in mind and the problem of constructing long codes that
can be decoded is particularly severe. This seems to be the main reason why
approaching the Shannon performance has proved to be so difficult.

1.11 CODING FOR MULTILEVEL MODULATIONS

The standard modulation for satellite communications is QPSK, but 8-PSK or 16-
PSK could be used to obtain 3 or 4 transmitted bits (respectively) per transmitted
symbol. Unfortunately, this results in reduced noise immunity. With m bits per
transmitted symbol, assuming that the energy per transmitted bit is maintained, the
energy per transmitted symbol can increase by a factor of m relative to binary PSK.
The distance between closest points in the constellation will, however, be propor-
tional to sin(7i/M), where M — 2m, as shown in Figure 1.9, and the noise energy
required to cause an error will depend on the square of this. The uncoded perform-
ance relative to binary PSK is therefore

G w (dB)=101og lo [msin 2 (7r/2"')]

The values are shown in Table 1.7.

As can be seen, there are severe losses associated with higher level constellations,
making coding all the more important. The codes, however, need to be designed
specifically for the constellation to maximize the distance in signal space, the Euclid-
ean Distance, between code sequences.

Figure 1.9 Distance between MPSK constellation points

Table 1.7 Performance of uncoded MPSK

m M Gm (dB)
1 2 0.0
2 4 0.0
3 8 -3.6
4 16 -8.2
5 32 -13.2
6 64 -18.4
7 128 -23.8
8 256 -29.2
22 ERROR CONTROL CODING

The principal approach to designing codes for this type of system is to take a
constellation with m bits per symbol and to use a rate (m - 1)/m code so that the
information throughput will be the same as the uncoded constellation with m - 1 bits
per symbol and the performances can be compared directly. Convolutional codes of
this type are known as Ungerboeck codes and will be described in Chapter 2.

1.12 CODING FOR BURST-ERROR CHANNELS

Coding performance curves are regularly shown for the AWGN channel. There are
two reasons why this is so. Firstly, burst-error mechanisms are often badly under-
stood and there may be no generally accepted models that fit the real behaviour. The
increasing importance of mobile communications where the channel does not re-
motely fit the AWGN model has, however, led to considerable advances in the
modelling of non-Gaussian channels. The other reason is that most codes in use
are primarily designed for random error channels. The only important codes where
this is not the case are Reed Solomon codes which are constructed with multibit
symbols and correct a certain number of symbol errors in each codeword. A burst of
errors affecting several bits close together may affect only a few symbols of the code
and be correctable, as shown in Figure 1.10. The symbols each consist of 4 bits and a
burst spanning 8 bits containing 5 errors has affected only 3 symbols.
For the most part, we shall be faced with trying to make a random bit-error-
correcting code work on a burst-error channel, and the technique that is used is
interleaving. Essentially, this consists of reordering the bits before transmission
(interleaving) and putting them back into the original order on reception (deinter-
leaving). As the error locations are affected only by the deinterleaving, they become
scattered through the code-stream so that they appear as random errors to the
decoder.
There are two main types of interleaving to consider, block interleaving and
convolutional interleaving. Both will be explained as if they are being used with a
block code, although both can be used with convolutional codes too.
Block interleaving is illustrated in Figure 1.11. Codewords are written into the
columns of an array and the total number of columns, A, is termed the interleaving
degree. If a burst of errors spans no more than A symbols, then there will be at most
one error in each codeword. A code that can correct up to / errors could correct, for
example, up to t bursts of length A, one burst of length (At or a mixture of shorter
bursts and random errors.
Convolutional interleaving is shown in Figure 1.12. The codewords in the columns
of the array are shifted through delays which differ for each symbol. Usually these
are increasing by one for each row of the array. The order of symbols on the channel

| HI |ii IK I 'tv Error location

Figure 1.10 Binary burst error on multibit symbols

 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 23

codewords

Code order
symbols

Transmission
order

Figure 1.11 Block interleaving

Jim ymm

Figure 1.12 Convolutional interleaving

follows the diagonal sequence shown. Any burst of errors will affect symbols in the
transmission stream as shown and it can be seen that the burst must exceed n + 1
symbols in length before it affects two symbols of the same codeword. If the delays
are increasing by D for each symbol, then the separation of two symbols from the
same codeword is Dn + 1. In effect this is the interleaving degree.
The main differences between the two types of interleaving are that the convolu-
tional interleaver will extend the symbol stream through the presence of null values in
the delay registers, but block interleaving will have more delay because of the need to
fill the array before transmission can commence.
One might think that the block interleaver would introduce a delay of In symbols,
however it is possible to start transmission a little before the array is filled. The
encoder must have (A — 1)« + 1 symbols prepared by the time that A symbols are
transmitted; otherwise, the rightmost symbol of the top row will not be ready in time
for transmission (assuming that symbol is transmitted the instant it is prepared). The
delay is therefore (A - 1)« + 1 - A = (A - \)(n - 1) symbols. The same delay will
occur in the deinterleaver which writes the symbols into rows and decodes by
column, giving an overall delay of 2(A — !)(« — 1) symbols.
The convolutional interleaver introduces D + 2D + h (n - !)D = n(n -1)D/2
dummy symbols into the stream. The deinterleaver applies a delay of (n – l)D to the
top row, (n - 2)D to the second row, etc., introducing the same number of dummy
symbols. The overall delay is therefore n(n — l)D. As the interleaving degree is
nD + 1, the overall delay is (A - \)(n - 1), half the value of the block interleaving.
24 ERROR CONTROL CODING

1.13 MULTISTAGE CODING

The aim of making an effective long code is sometimes approached by multistage
coding in which the overall code is constructed from simple components, thus provid-
ing a feasible approach to decoding. Examples of this type of approach include serial
concatenation, in which information is first encoded by one code, the outer code, and
then the encoded sequence is further encoded by a second code, the inner code. Reed
Solomon codes are often used as outer codes because of their ability to correct the
burst errors from the inner decoder. Another approach is the product code in which
information is written into an array and the rows and columns are separately encoded.
In recent years other types of concatenation have become of interest in conjunction
with iterative decoding techniques, where decoding of the second code is followed by
one or more further decodings of both codes. In particular, iterative decoding is
applied to parallel concatenated codes, namely the application of two systematic
codes to a single-information stream to derive two independent sets of parity checks.
This is the principle of the so-called turbo codes and other similar constructions,
which are treated in Chapter 10.

1.14 ERROR DETECTION BASED METHODS

So far we have assumed that the intention is to correct all errors if possible; this is
known as Forward Error Correction (FEC). We have, however, seen that detected
uncorrectable errors are possible. In fact there may be good reasons not to attempt
error correction provided we have some other way of dealing with erroneous data.
Not attempting error correction will not make the maximum use of the received
sequence, but it makes it less likely that there will be undetected errors and reduces
the complexity at the receiver.
There are two main possibilities if errors are not to be corrected. The first
approach is to use a reverse channel (where available) to call for retransmission.
This is known as Retransmission Error Control (REC) or Automatic Retransmission
reQuest (ARQ). The second approach is to process the data in such a way that the
effect of errors is minimized. This is called Error Concealment.

1.14.1 ARQ strategies

The transmitter breaks the data into frames, each of which contains a block code
used for error detection. The receiver sends back acknowledgements of correct
frames and whenever it detects that a frame is in error it calls for retransmission.
Often the transmitter will have carried on sending subsequent frames, so by the time
it receives the call for retransmission (or fails to obtain an acknowledgement within a
predefined interval) it will already have transmitted several more frames. It can then
either repeat just the erroneous frame (Selective Repeat ARQ) or else go back to the
point in the sequence where the frame error occurred and repeat all frames from that
point regardless (Go Back N ARQ).
 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 25

If Selective Repeat (SR-ARQ) is employed, the receiver must take responsibility

for the correct ordering of the frames. It must therefore have sufficient buffering to
reinsert the repeated frame into the stream at the correct point. Unfortunately, it is
not possible to be sure how many repeats will be needed before the frame will be
received. The protocols therefore need to be designed in such a way that the trans-
mitter recognizes when the receiver's buffer is full and repeats not only erroneous
frames but also those which will have been lost through buffer overflow.
Neglecting effects of finite buffers, assuming independence of errors from frame to
frame and a frame error rate of pf, the efficiency of SR-ARQ is

where n is the total frame length and k is the amount of information in the frame. The
difference between n and k in this case will not be purely the parity checks of the code.
It will include headers, frame numbers and other fields required by the protocol.
For Go Back N (GBN-ARQ), there is no need for receiver buffering, but the
efficiency is lower. Every time x frames are received correctly, followed by one in
error, the transmitter goes on to frame x + N before picking up the sequence from
frame x + 1 . We can therefore say that

Now the probability of x frames being successful followed by one that fails is
pf(1 — p f ) x ; therefore

x =£>/(! -Pfi=pf(\ -/>/)[! + 2(1 -Pf) + 3(1 -pf)2 + •••}

i=0

The sum to infinity of the series in the square brackets is l/pj-, so we find that

Pf
Hence

It may appear from this that an efficient GBN scheme would have a small value of
N, however the value of N depends on the round trip delays and the frame length.
Small values of N will mean long frames which in turn will have a higher error rate.
In fact it is the frame error rate that is the most important term in the efficiency
expression, with the factor k/n also playing its part to ensure that frames cannot be
made too small.
26 ERROR CONTROL CODING

The main difficulties with ARQ are that efficiency may be very low if the frame
error rate is not kept low and that the delays are variable because they depend on the
number of frame errors occurring. The delay problem may rule out ARQ for real
time applications, particularly interactive ones. The solution to the efficiency prob-
lem may be to create some sort of hybrid between FEC and ARQ with FEC
correcting most of the errors and reducing the frame error rate and additional
error detection resulting in occasional use of the ARQ protocols.

1.14.2 Error concealment

Some applications carry data for subjective appreciation where there may still be
some inherent redundancy. Examples include speech, music, images and video. In
this case, the loss of a part of the data may not be subjectively important, provided
that the right action is taken. Designing a concealment system is a signal processing
task requiring knowledge of the application, the source coding and the subjective
effects of errors. Possibilities include interpolation or extrapolation from previous
values. Hybrids with FEC are also possible.
Error concealment is often appropriate for exactly the applications where ARQ is
difficult or impossible. One example is digital speech where the vocoders represent
filters to be applied to an input signal. The filter parameters change relatively slowly
with time and so may be extrapolated when a frame contains errors. Another
example occurs with music on compact disc where the system is designed in a way
that errors in consecutive samples are unlikely to occur. The FEC codes have a
certain amount of extra error detection and samples known to contain errors are
given values interpolated from the previous and the following sample.

1.14.3 Error detection and correction capability of block codes

Error detection schemes or hybrids with FEC are usually based on block codes. In
general, we can use block codes either for error detection alone, for error correction or
for some combination of the two. Taking into account that we cannot correct an error
that cannot be detected, we reach the following formula to determine the guaranteed
error detection and correction properties, given the minimum distance of the code:
dmin>S+t (1.10)

where s is the number of errors to be detected and t ( < s) is the number of errors to
be corrected. Assuming that the sum of s and t will be the maximum possible then

dmin = S + t + 1

Thus if dmin = 5, the possibilities are

s=4 t =0
5=3 t=1
5=2 t=2
 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 27

If we decided, for example, to go for single-error correction with triple-error

detection, then the occurrence of four errors would be detected, but the likelihood
is that the decoder would assume it was the result of a single error on a different
codeword from the one transmitted.
If the code is to be used for correction of the maximum amount of errors, and if the
value of minimum distance is odd, then setting t = s gives

dmin=2t+1 (1.11)

1.15 SELECTION OF CODING SCHEME

The factors which affect the choice of a coding scheme are the data, the channel and
specific user constraints. That includes virtually everything. The data can have an
effect through its structure, the nature of the information and the resulting error-rate
requirements, the data rate and any real-time processing requirements. The channel
affects the solution through its power and bandwidth constraints and the nature of
the noise mechanisms. Specific user constraints often take the form of cost limita-
tions, which may affect not only the codec cost but also the possibility of providing
soft-decision demodulation.

1.15.1 General considerations

The major purpose of incorporating coding into the design of any system is to reduce
the costs of the other components. Reliable communications can usually be obtained
by simple, yet costly, methods such as increasing power. A well-designed coding
scheme should result in a lower overall system cost for an equivalent or better
performance. If this objective is to be met, however, the designer needs to make a
careful choice and be aware of the whole range of available techniques.
Convolutional codes are highly suitable for AWGN channels, where soft decisions
are relatively straightforward. The coding gains approach the asymptotic value at
relatively high bit error rates, so that at bit error rates of 10–5 to 10–7 in Gaussian
conditions, convolutional codes are often the best choice. Many types of conditions,
however, can give rise to non-Gaussian characteristics where the soft-decision thresh-
olds may need to adapt to the channel conditions and where the channel coherence
may mean that Viterbi decoding is no longer the maximum likelihood solution. The
complexity of the decoder also increases as the code rate increases above 1/2, so that
high code rates are the exception. Even at rate 1/2, the channel speed which can be
accommodated is lower than for Reed Solomon codes, although it is still possible to
work at over l00 Mbits / second, which is more than enough for many applications!
Reed Solomon codes have almost exactly complementary characteristics. They do
not generally use soft decisions, but their performance is best in those conditions where
soft decisions are difficult, i.e. non-Gaussian conditions. In Gaussian conditions the
performance curves exhibit something of a 'brick wall' characteristic, with the codes
28 ERROR CONTROL CODING

working poorly at high bit error rates but showing a sudden transition to extremely
effective operation as the bit error rate reduces. Thus they may show very high
asymptotic coding gains but need low bit error rates to achieve such gains. Conse-
quently they are often advantageous when bit error rates below 10–10 are required.
Error rates as low as this are often desirable for machine-oriented data, especially if
there is no possibility of calling for a retransmission of corrupted data. The decoding
complexity reduces as code rate increases, and in many cases decoding can be
achieved at higher transmitted data rates. They can also, of course, be combined
with other codes (including convolutional codes or other RS codes) for concatenated
coding.
For the future, the so-called turbo codes are going to be of increasing importance.
These are tree codes of infinite constraint length, used in combination and decoded by
an iterative method. Usually two codes are used with one operating on an interleaved
data set. The decoding algorithms not only use soft decisions, they also provide soft
decisions on the outputs, and the output of each decoder is fed to the input of the other
so that successive iterations converge on a solution. The performance is extremely
good, giving acceptable error rates at values of Eb/ No little above the Shannon levels.
There are, however, several problems to be resolved including the existence of an error
floor making it difficult to achieve output BERs below 10–5or 10–6.
The above considerations certainly do not mean that other types of codes have no
place in error control. Many considerations will lead to the adoption of other
solutions, as will be seen from the discussions below. Nevertheless, mainstream
interests in future systems are likely to concentrate on Viterbi-decoded convolutional
codes, Reed Solomon codes and turbo codes, and the designer wishing to adopt a
standard, 'off-the-shelf solution is most likely to concentrate on these alternatives.

1.15.2 Data structure

If information is segmented into blocks, then it will fit naturally with a block coding
scheme. If it can be regarded as a continuous flow, then convolutional codes will be
most appropriate. For example, protecting the contents of computer memories is
usually done by block coding because the system needs to be able to access limited
sections of data and decode them independently of other sections. The concept of
data ordering applies only over a limited span in such applications. On the other
hand, a channel carrying digitized speech or television pictures might choose a
convolutional scheme. The information here is considered to be a continuous stream
with a definite time order. The effects of errors will be localized, but not in a way
which is easy to define.
It is important to separate the structure of the data from the characteristics of the
channel. The fact that a channel carries continuous data does not necessarily mean
that the data is not segmented into block form. Less obvious, but equally important,
a segmented transmission does not necessarily imply segmented data. A TDMA
channel, for example, may concentrate several continuous streams of information
into short bursts of time, but a convolutional code may still be most appropriate.
With adequate buffering, the convolutional code on any stream may be continued
across the time-slots imposed by the TDMA transmission.
 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 29

1.15.3 Information type

It is conventional to assess the performance of coding schemes in terms that involve

bit error rates. This is not really appropriate for many types of information, and the
most appropriate measure will often affect the choice of a coding scheme. Indeed it is
difficult to think of any application in which the bit error rate is directly important. If
discrete messages are being sent, with every bit combination representing a totally
different message, then the message error rate is of crucial importance; the number of
bit errors in each wrong message is not important at all. Even with information that
is subjected to some kind of sensory evaluation (i.e. it is intended for humans, not for
machines), not all bits are equal. In most cases there are more and less significant bits
or some bits whose subjective importance is different from that of others. Digitized
speech without any data compression carries a number of samples, each of which has
a most and a least significant bit. Only if bit errors in all positions have equal effect
will bit error rate provide a measure of subjective quality. If the speech is at all
compressed, the bits will represent different types of information, such as filter poles
or excitation signals, and the subjective effects will vary. Data intended for subjective
evaluation may be suitable for error concealment techniques.
Errors on a coded channel can be placed into four categories. There are those
which are corrected by the code and allow the information to be passed on to the
destination as if those errors had never occurred. There are errors which are detected
but not corrected. There are also errors which are not detected at all and errors which
are detected but the attempted correction gives the wrong result. Errors are passed on
to the destination in the last two cases. For many applications it is important to
minimize the probability of unsuspected errors in the decoder output. This will bias
the user towards block codes, which often detect errors beyond the planned decoding
weight, and away from forward error correction which accepts that undetected
decoding errors will occur. The strength of the bias depends on the consequence of
errors. If an error could start the next world war, it is obviously of more importance
than one that causes a momentary crackle on a telephone line.
Acceptable error rates will depend not only on the type of data but also on whether
it will be processed on- or off-line. If data is to be processed immediately, it may be
possible to detect errors and invoke some other strategy such as calling for retrans-
mission. Off-line processing means that errors cannot be detected until it is too late to
do anything about it. As a result the error rate specification will commonly be lower.
Note that there must always be some level of errors which is considered to be
acceptable. It is easy to set out with a goal of eliminating all errors. Achieving this
goal would require infinite time and an infinite budget.

1.15.4 Data rate

It is difficult to put figures on the data rates achievable using different codes. This is
partly because any figures given can quickly become out of date as technology
advances and partly because greater speeds can usually be achieved by adopting a
more complex, and therefore more expensive, solution. Nevertheless, for a fixed
complexity, there are some codes which can be processed more rapidly than others.
30 ERROR CONTROL CODING

The codes which can be processed at the highest data rates are essentially simple,
not very powerful, codes. Examples are codes used purely for error detection.
Concatenated codes using short block inner codes are not far behind because the
computations on the Reed Solomon codes are done at symbol rate, not bit rate, and
the block codes used are extremely simple. It follows that Reed Solomon codes alone
are in the highest data rate category. Viterbi-decoded convolutional codes are fast
provided the input constraint length is not too long, say no more than 9. BCH codes
can also be used at similar rates provided hard-decision decoding only is required.
Soft-decision decoding of block codes and the more complex concatenated schemes,
e.g. turbo codes, are capable of only moderate data rates.
Of course, the required data rate affects the choice of technology too; the more that
can be done in hardware the faster the decoding. Parallelism can increase decoding
speeds, but with higher hardware complexity and therefore cost. A data rate of a few
thousand bits per second could allow a general-purpose microprocessor to be used
for a wide range of codecs, but obviously that would be uneconomic for volume
production. Many of the influences of data rate on system design will be closely
bound up with economics.

1.15.5 Real time data processing

If real time data processing is required, the decoder must be able to cope with the link
data rates. This may be achieved at the expense of delays by, for example, decoding
one sequence while the next is being buffered. The decoding delay may in some cases
become significant, especially if it is variable.
Forward error correction requires a decoding delay that, in most cases, depends on
the exact errors which occur. Nevertheless, there is usually a certain maximum delay
that will not be exceeded. Buffering the decoded information until the maximum
delay has expired can therefore produce a smooth flow of information to the
destination. Two major factors determining the delay will be the data rate and the
length of the code. Information theory tells us that long codes are desirable, but for
many applications long delays are not. Thus the maximum acceptable delay may
limit the length of the codes that can be used.
If no maximum decoding delay can be determined, then the decoded information
will come through with variable delays, which can cause havoc with real time infor-
mation. The main error control strategy that exhibits variable delays is ARQ because
one cannot guarantee that any retransmission will be successful. These problems may
be minimized by the use of a suitable ARQ / FEC hybrid.

1.15.6 Power and bandwidth constraints

These constraints drive the solution in opposite directions. In the absence of band-
width constraints one would use a low rate concatenated code to achieve high coding
gains or very low error rates. Very tight bandwidth constraints, making binary
modulation incompatible with the required data rate and error rates, require the use
of specially designed codes in conjunction with multilevel modulations. Traditionally
 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 31

these have been convolutional codes, but block codes may be possible and turbo-coded
solutions are being developed.
Assuming that the major aim of coding is to reduce the power requirement for
a given error rate, high coding gains would appear to be desirable. There can be
no doubt that the highest gains are achievable using turbo codes or concatenated
codes. If the gain requirement is less stringent, convolutional codes with hard-
decision sequential decoding or soft-decision Viterbi decoding (to be described in
Chapter 2) provide the highest gains on a Gaussian channel.

1.15.7 Channel error mechanisms

Ideally one would design a coding scheme for the precise conditions encountered on
the channel. In practice, the channel may not be well characterized and the coding
scheme may have to show flexibility to cope with the range of possible conditions.
For slowly varying channel conditions which exhibit approximate Gaussian condi-
tions over appreciable periods of time, adaptive coding schemes are a natural choice.
These often use variable rate convolutional codes, or they may be based around
ARQ / FEC hybrids. For channels which may fluctuate rapidly between states,
producing mixtures of bursty and random errors, a wide variety of diffuse-error
correcting schemes, including interleaving, are available. Reed Solomon codes may
also be considered to fall into this category; although optimized neither for random
errors or general bursts, their low redundancy overhead makes them a good choice
for a variety of channel conditions.

1.15.8 Cost

Any error control scheme is merely a part of a larger system and its costs must be in
proportion to its importance within the system. Bearing in mind that error rates may
be reduced by the use of higher transmitted power, the aim of coding is to be more
cost-effective than other solutions. That, however, is often not the main way in which
cost constraints are experienced in a coding system; the major part of the costs of
error control are incurred at the decoder, placing the burden of the economics onto
the receiving equipment. Since the owners of transmitting and receiving equipment
may be different, the economic considerations may not be combined to optimize
overall system costs. Decoder costs must be assessed in terms of what the receiver will
be prepared to pay.
A number of fairly straightforward rules may be stated. Firstly as previously
indicated, the decoder costs dominate in a forward error correction scheme. Error
detection is therefore much cheaper than error correction. High data rates will cost
more than low data rates. Complex codes with multiple-error correction will
cost more than simpler codes. For many applications, however, the main factor
affecting cost will be whether there is a codec available commercially or whether it
will need to be developed specially. Development costs must be spread across the
number of receivers, and if the market is small or particularly cost-sensitive it may be
32 ERROR CONTROL CODING

impossible to develop a special codec for the particular needs of the application. In
that case, the choice will be severely limited.
Any very specific advice about commercially available codecs would ensure that
this book would quickly be out of date. As with all modern technologies, the product
range is expanding and getting cheaper. Rate 1 /2 Viterbi decoders are available and
popular, and may incorporate puncturing for higher rates or for adaptable coding
schemes (although the latter involve many other system complexities and costs).
Certain Reed Solomon codes are being adopted as standard and codecs are becoming
available. Often this will be a spin-off from a particular mass market, such as
compact disc players.
Although it seems a shame to sound a negative note, I believe that many interesting
ideas in error control will never be implemented simply because their potential market
will not make the development costs worthwhile. Similarly many engineers working on
error control techniques will never be allowed to design the best system technically;
they will be forced to choose the best of what is available. Those who wish to have a
relatively free hand should work on applications where the potential market is large or
not very cost-sensitive. The same constraints apply, of course, in many other areas.
Some would say that is what engineering is all about and error control is, after all, an
engineering topic rather than a mathematical one. The mathematics is the servant of
the engineer, and the engineering is the really difficult part.

1.16 CONCLUSION
In this chapter we have been concerned mainly with general concepts and back-
ground. There are several good modern books on digital communications that
include a treatment of error control codes [2–4]. These can be consulted for more
information about digital modulations implementation issues and applications. They
can also be used to provide an alternative view of error control coding issues that will
be treated later in this book. Another large subject given very brief treatment here is
the general topic of information theory, and other sources [5, 6] are recommended for
further reading.
The next chapter of this book deals with convolutional codes, which are the most
commonly adopted codes in digital communications. Chapter 3 will cover linear
block codes and a subset of these codes, cyclic codes, will be treated in Chapter 4.
The construction of cyclic codes and the decoding methods for multiple-error correc-
tion require a knowledge of finite field arithmetic, and this is covered in Chapter 5.
Chapter 6 then deals with BCH codes, a large family of binary and nonbinary codes,
but concentrating on the binary examples. The most important nonbinary BCH
codes are Reed Solomon codes and these are treated in Chapter 7. Chapter 8 then
deals with performance issues relevant to all block codes. Multistage coding is
introduced in chapter 9. Codes using soft-in-soft-out algorithms for iterative decod-
ing are covered in Chapter 10.
It should not be assumed that the length of the treatment of different codes
indicates their relative importance. Block codes have a very strong theoretical
 THE PRINCIPLES OF CODING IN DIGITAL COMMUNICATIONS 33

basis, but for many applications the chapters on convolutional codes and on iterative
decoding will be the most relevant. Iterative decoding can, however, be applied to
block codes and familiarity with Chapters 4 and 9 will certainly help in obtaining the
most from Chapter 10.

1.17 EXERCISES
1 A transmitter sends one of four possible messages. The most likely occurs with a
probability of 0.5 and is represented by the value 0. The second most likely occurs
with a probability of 0.25 and is given the representation 10. The other two
messages each occur with a probability of 0.125 and are given the representations
110 and 111. Find the mean information content of the messages and the mean
number of bits transmitted. Compare with the case where a fixed length repre-
sentation is used.

2 Find the value of Er /N0 needed by BPSK or QPSK modulation to achieve a bit
error rate of 10 –3 over an AWGN channel.

3 Find approximate optimum uniform spacing for 16–level quantization from a

BPSK or QPSK receiver, assuming an AWGN channel with Er / N0 = 2dB.

4 Use Table 1 . 1 to carry out minimum distance decoding of hard-decision sequences

0 1 0 1 1 , 0 1 1 1 0, 1 0 1 00, 1 1 000. Use the method of Section 1 . 8 . 3 to check the results .

5 Prove that, for a linear code, the distance structure is the same viewed from any
codeword.

6 An uncoded channel needs Eb/ N0 of 2 dB to achieve a BER of 10–2 and 10 dB to

achieve a BER of 10–5. A rate 1/ 2 code subject to random bit errors with probabil-
ity of 0.01 produces an output BER of 10–5. What is the coding gain at 10–5 BER? If
the coded channel operates at Eb/ N0 = 5 dB, what is the uncoded BER?

7 Find the maximum coding gain that could be achieved for BER = 10–6 using
QPSK modulation over an AWGN channel and a rate 1/3 code.

8 A block code has dmin = 8. Find the maximum guaranteed error detection if
maximum error correction is to be carried out. How would this change if only
single-error patterns were to be corrected? Find the amount of error correction
achievable if an extra three bits in error should be detectable.

9 A source transmits a sequence of numbered frames, inserting repeats of previous

frames as required for ARQ. The second, sixth and ninth frames transmitted are
corrupted on reception. Show the flow of frames between the transmitter and
receiver to support a GB3 protocol.
34 ERROR CONTROL CODING

10 Assuming the same propagation delays and the same corrupted frames as
question 9, find how many frame intervals would be needed to transfer 7 frames
if SR-ARQ were used, assuming
(a) an infinite receiver window
(b) a minimum receiver window to operate SR-ARQ.
Quantify the receiver window size needed for part (b).

1.18 REFERENCES
1 J. Massey, Coding and Modulation in Digital Communications, Proceedings of the Inter-
national Zurich Seminar on Digital Communications, 1984.
2 S. Haykin, Communication Systems, 4th edition, John Wiley & Sons, 2001.
3 B. Sklar, Digital Communications: fundamentals and applications, 2nd edition, Prentice
Hall, 2001.
4 J.G. Proakis, Digital Communications, 4th edition, McGraw Hill, 2001.
5 S.W. Golomb, R.E. Peile and R.A. Scholtz, Basic Concepts in Information Theory and
Coding, Plenum Press, 1994.
6 R.E. Blahut, Principles and Practice of Information Theory, Addison Wesley, 1987.
2
C onvolutional codes

2.1 INTRODUCTION
In Chapter 1 it was explained that codes for error control generally fell into two
categories, namely block codes and convolutional codes. Many telecommunications
applications have used convolutional codes because of their ability to deliver good
coding gains on the AWGN channel for target bit error rates around 10–5.
As background to this chapter most of the first chapter is relevant, even though the
examples in Chapter 1 were based on a block code. The main decoding method,
Viterbi decoding, is indeed an implementation of the maximum likelihood decoding
that was explained there. This means that the discussion on soft-decision decoding
will be relevant as it will almost always be implemented in conjunction with Viterbi
decoding. Convolutional codes are also linear, as previously defined.

2.2 GENERAL PROPERTIES OF CONVOLUTIONAL

CODES
An example of the schematic diagram of a convolutional encoder is shown in Figure
2.1. Note that the information bits do not flow through directly into the code-stream,
i.e. the code is not systematic. Nonsystematic codes give better performance than
systematic codes when Viterbi decoding is used.
The way in which the encoder works is that the input bit is modulo-2 added to
stored values of previous input bits, as shown in the diagram, to form the outputs

input

Figure 2.1 Convolutional encoder

Exploring the Variety of Random
Documents with Different Content
is represented, as saying, In all my labours, they shall find none
iniquity in me that were sin, Hos. xii. 8. whereas God says in a
following verse, that they provoked him to anger most bitterly, ver.
14. and, after this, they entertain favourable thoughts of the vilest
actions, as some are represented doing, Who call evil good, and
good evil; that put darkness for light, and light for darkness; that put
bitter for sweet, and sweet for bitter, Isa. v. 20.
4. The next spiritual judgment mentioned in this answer, as a
punishment for sin, is a person’s being given up to vile affections.
This God is said to have done, to those whom the apostle describes,
as giving themselves over to the committing of those sins, which are
contrary to nature, Rom. i. 26. such as all men generally abhor, who
do not abandon themselves to the most notorious crimes: This is a
contracting that guilt, which is repugnant to those natural ideas of
virtue and vice, which even an unregenerate man, who has not
arrived to this degree of impiety, cannot but abhor. These are such
as are not to be named among Christians, or thought of, without the
utmost regret, and an afflictive sense of the degeneracy of human
nature.
5. The last thing mentioned in this answer, in which the inward
punishment of sin, in this life, consists, is, Horror of conscience.
Under the foregoing instances of spiritual judgments, conscience
seemed to be asleep, but now it is awakened, and that by the
immediate hand of God, and this is attended with a dread of his
wrath falling upon it: horror and despair are the result hereof; The
arrows of the Almighty are within him, the poison whereof drinketh
up his spirit; the terrors of God do set themselves in array against
him, Job vi. 4. and, Terrors take hold on him as waters; a tempest
stealeth him away in the night. The east wind carrieth him away,
and he departeth; and, as a storm, hurleth him out of his place. For
God shall cast upon him, and not spare; he would fain flee out of his
hand, chap. xxvii. 20-22.
This differs from those doubts and fears, which are common to
believers, inasmuch as it is attended with despair, and a dreadful
view of God, as a God to whom vengeance belongeth, and is
attended, as the apostle says, with a certain fearful looking for of
judgment, and fiery indignation, which shall devour the adversaries,
Heb. x. 27. Before this, he took a great deal of pains to stifle
convictions of conscience, but now he would fain do it, but cannot;
which is a sad instance of the wrath of God pouring forth gall and
wormwood into it, when he says, to use the prophet’s words, Thine
own wickedness shall correct thee, and thy backslidings shall
reprove thee, Jer. ii. 19.
But, now we are speaking concerning horror of conscience, we must
take heed, lest we give occasion to doubting believers, who are
under great distress of soul, through a sense of sin, to apply what
has been said, to themselves, for their farther discouragement, and
conclude, that this is a judicial act of God, and a certain evidence,
that they have not the truth of grace: Therefore we may observe,
that there is a difference between this horror of conscience, which
we have been describing, and that distress of soul, which believers
are often liable to, in three respects.
(1.) The former, under horror of conscience, flee from God, as from
an enemy, and desire only to be delivered from his wrath, and not
from sin, the occasion of it; whereas the believer desires nothing so
much, as that his iniquity, which is the occasion of it, may be
subdued and forgiven, and that he may have that communion with
God which he is destitute of; and, in order thereunto, he constantly
desires to draw nigh to him in ordinances, and, if he cannot enjoy
him he mourns after him: Thus the Psalmist complaineth, as one in
the utmost degree of distress, Thy wrath lieth hard upon me, and
thou hast afflicted me with all thy waves, Psal. lxxxviii. 7. yet he
says, Unto thee have I cried, O Lord, and in the morning shall my
prayer prevent thee, ver. 13.
(2.) The one reproaches God, and entertains unworthy thoughts of
him, as though he were severe, cruel, and unjust to him; whereas
the other, with an humble and penitent frame of spirit, complains
only of himself, acknowledges that there is no unrighteousness with
God, and lays all the blame to his own iniquity.
(3.) Horror of conscience, when it is judicial, seldom continues any
longer, than while a person is under some outward afflictive
dispensation of providence, under which sin is increased, and the
removal thereof leaves him as stupid as he was before: whereas it is
otherwise with a believer; for the removal of God’s afflicting hand, as
to outward troubles, will not afford him any remedy against his
fears, unless sin be mortified, and God is pleased to lift up the light
of his countenance upon him, and give him joy and peace in
believing.
Secondly, Having considered the inward punishments of sin in this
life we are now to speak something concerning those, which, in this
answer, are styled outward, of which some are the immediate
consequence of the first entrance of sin into the world, and others
are increased by the frequent commission thereof; the former
includes in it the curse of God upon the creature for our sakes, and
our liableness to death; the latter respects those various other evils
that befal us, of which some are personal, and others relative;
accordingly, many evils are said to befal us, in our bodies, names,
estates, relations, and employments.
1. The curse of God was denounced against the creatures,
immediately after man’s apostasy from him: This is, in part,
contained in the threatning, Cursed be the ground for thy sake.
Thorns and thistles shall it bring forth to thee; by the sweat of thy
face shalt thou eat bread, till thou return to the ground, Gen. iii. 17-
19. and it is very elegantly described by the apostle, who speaks
of[83] the creature as subject to vanity, not willingly but by reason of
him, who hath subjected the same in hope;[84] and of the whole
creation’s groaning and travelling in pain together until now, Rom.
viii. 20-22. the general scope and design whereof seems to be this,
that it retains the visible marks of the curse of God, which followed
upon man’s sin. This I rather think to be the sense thereof, than to
suppose, as some do, that the creature, here spoken of, is the
Gentile world, and the vanity, which they were subject to, that
idolatry which they were universally addicted to; for that does not
seem to agree with what the apostle says, when he supposes that
their subjection to this vanity was not willingly, neither can it well be
called the bondage of corruption. But if, on the other hand, we take
it for that part of the creation, which was more immediately
designed for the use of man, being abused, and so subject to that
vanity, which is the consequence of his fall, this agrees very well
with its being not willingly; for he is speaking here of creatures not
endowed with understanding and will, yet abused by those that are,
and therefore their subjection to man’s vanity, is not so much from
themselves, as from man’s sin; and then he speaks of the liableness
of all these things to corruption, as the world is decaying and
growing toward a dissolution. How far this curse of God, on the
creature, extended itself, whether only to this lower world, or to the
heavenly bodies themselves, such as the sun, moon, and stars, I
pretend not to determine; for I desire not to extend my conjectures
beyond the line of scripture, which speaks of the earth, as cursed for
man’s sake; and how far the other parts of nature, are liable to
corruption, or inclined towards a dissolution, it is hard to say. All that
I shall add, on this head, is, that, when this is called a punishment,
which is consequent on man’s sin, it more especially respects man,
who is the only subject of punishment in this world: inanimate
creatures are the matter, in which he is punished, but he alone is the
subject thereof.
2. There are other evils that befal us, in which we are more
immediately concerned, and these are either personal or relative;
and, accordingly,
(1.) We are liable to bodily diseases, which are a continual
weakness, or decay of nature; and afterwards to death, which is the
dissolution of the frame thereof. All the pains and disorders of
nature, whereby our health is impaired, and our passage, through
this world, rendered uneasy, are the consequence of our sinful and
fallen state, and, in that respect, are sometimes styled, a
punishment of sin: thus, when our Saviour healed the man that was
sick of the palsy, he intimates, that his sickness was the
consequence of sin, by the mode of expression used, Thy sins are
forgiven thee, Mat. ix. 2. and the Psalmist speaks of God’s pardoning
the iniquities of his people, and healing all their diseases, Psal. ciii.
3. at the same time; in this respect, they are styled, in a more large
sense, a punishment of sin: but, when they have a mixture of the
wrath of God in them, and are not rendered subservient to our good,
nor included among those dispensations, which are called fatherly
chastisements, as they are not in those that are in an unjustified
state, they are, in a more proper sense, punishments of sin. Thus
the diseases that God brought on the Egyptians, are reckoned
among the plagues of Egypt, and so were a visible instance of the
vindictive justice of God. The same thing may be said of death,
which is the dissolution of the frame of nature, which is a
consequence of sin, in all, and in the most proper sense, a
punishment of sin, in those, who are liable not only to the stroke,
but the sting of death, and thereby are brought under the power of
the second death.
(2.) There are many evils that befal us in our names, when we meet
with reproaches and injurious treatment, as to what concerns our
character in the world, from those who act as though their tongues
were their own, and they were not accountable to God, for those
slanders and revilings, which they load us with. We are, in this case,
very ready to complain of the injustice done us, by their
endeavouring to deprive us of that, which is equally valuable with
our lives: but we ought to consider, that sin is the cause of all this,
and God’s suffering them thus to treat us, and thereby to hinder our
usefulness in the world, must be reckoned a punishment of sin.
(3.) There are other evils that befal us in our secular concerns,
namely, our estates and employments in the world, which are
entirely at the disposal of providence, which renders us rich, or poor,
succeeds, or blasts, our lawful undertakings. This God may do, out
of his mere sovereignty, without giving an account of his matters to
any one. But yet, when we meet with nothing but disappointments,
or want of success in business, and whatever diligence, or industry,
we use, appears to be to no purpose, and adverse providences, like
a torrent, sweep away all that we have in the world, and poverty
comes upon us, like an armed man, this is to be reckoned no other
than a punishment of sin.
(4.) There are other evils, which we are exposed to, in our relations,
by which we understand, the wickedness of those who are nearly
related to us, or the steps they take to ruin themselves, and cast a
blemish on the whole family to which they belong. The bonds of
nature, and that affection, which is the result thereof, render this
very afflictive: and especially when they, who are related to us,
attempt any thing against us to our prejudice, this is a circumstance
that sharpeneth the edge of the affliction. And, as it is a sin in them,
which is contrary to the dictates of nature; so sometimes we may
reckon it a punishment which we are liable to, as the consequence
of our sin in general. But, if we have occasion to reflect on our
former conversation, as not having filled up every relation with those
respective duties, that it engages to; if we have been undutiful to
our parents, or unfaithful servants to our masters, or broke the
bonds of civil society, by betraying or deserting our friends, and
setting aside all those obligations which they have laid us under; this
oftentimes exposes us to afflictive evils of the like nature, whereby
the affliction we meet with in others, appears to be a punishment of
our own sin. Thus concerning the punishment of sin in this life; from
whence we may make the following remarks.
1. Whatever evils we are exposed to in this world, we ought to be
very earnest with God, that he would not give us up to spiritual
judgments. The punishments of sin, which are outward, may be
alleviated and sweetened with a sense of God’s love, and made
subservient to our spiritual and eternal advantage. But blindness of
mind, hardness of heart, and those other evils, which tend to vitiate
and defile the soul, which have in them the formal nature of
punishment, these are to be dreaded like hell; and, as we are to be
importunate with God to prevent them, so we ought to watch
against those sins that lead to them; and therefore let us take heed
of being insensible, or stupid, under any afflictive evils, as neglecting
to hear the voice of God, who speaks by them, or refusing to receive
instruction by correction.
2. Let us not be too much dejected, or sink under those outward
afflictive providences, which we are liable to; for, though they be the
consequence of sin, yet, if we have ground to conclude, by faith,
that our sins are forgiven, they are not to be reckoned the stroke of
justice, demanding satisfaction, and resolving never to remove its
hand from us, till we are consumed thereby; since believers often
experience, what the prophet prays for, that God in wrath
remembers mercy, Hab. iii. 2.
3. Let us take heed that we do not ascribe afflictive providences to
chance, or content ourselves with a bare reflection on them, as the
common lot of man in this world, who is born to trouble as the
sparks fly upwards: For, this we may do, and not be humbled for
that sin, which they are designed to bring to remembrance, as they
are to be reckoned a punishment thereof.
4. Let us not murmur, or quarrel with God, as though he dealt hardly
with us, in sending afflictive evils; but rather let us bless him, how
heavy soever they appear to be, that they are not extreme, but
mitigated, and have in them a great mixture of mercy. Thus God
says, concerning the evils that he had brought upon Israel, that in
measure he would debate with them, who stayeth his rough wind in
the day of the east wind: and by this shall the iniquity of Jacob be
purged, Isa. xxvii. 8, 9. and, by this means, God not only afflicts us
less than our iniquities deserve, but brings good to us thereby in the
end. If the guilt of sin is taken away, we have ground to conclude,
that all these things shall work together for good, as he has
promised they shall, to those that love him. This leads us to
consider,
II. The punishment of sin in the world to come. Though the wrath of
God be revealed, in many instances, in a very terrible manner, as a
punishment of sin in this life, yet there is a punishment unspeakably
greater, which sinners are liable to, in the world to come. That this
may appear, let us consider the following propositions.
1. That the soul exists after its separation from the body by death;
which is evident, from the immateriality thereof, and its being of a
different nature from the body. This was known and proved by the
light of nature; so that the very heathen, who had no other light
than that to guide them, discover some knowledge of it. But this is
more plain from scripture; as when it is said, Fear not them which
kill the body, but are not able to kill the soul; but rather fear him,
which is able to destroy both soul and body in hell, Matt. x. 28.
2. The soul thus existing, though separate from its body, must be
supposed to retain those powers and capacities it had, while united
to it, which are proper to it, as a spirit, and particularly as the
subject of moral government; and those powers and capacities may
also be supposed to be in it in a greater degree, when dislodged
from the body, which is a great hindrance to it in its actings, as
every one sensibly experiences; therefore it follows,
3. That it cannot but be happy, or miserable, in another world; for
there is no middle state between these two. This is farther evident
from what was observed in the last proposition, concerning the
continuance and increase of its powers and faculties, whereby it is
rendered more capable thereof, than it is now.
4. If it goes out of this world, under the weight and guilt of sin upon
it, it must retain that guilt, because there is no sacrifice for sin,
extending itself to that world; no mediator, no gospel, or means of
grace; no promises of, or way to obtain forgiveness; therefore,
5. Wicked men, whose sins are not forgiven in this world, are the
subjects of punishment in the other.
6. This punishment cannot be castigatory, or paternal, or consistent
with the special love of God, or, for their advantage, as the
punishments of the sins of believers are in this world, since it is
always expressed as the stroke of vindictive justice, demanding
satisfaction for sins committed.
7. Some are happy in a future state, namely, those who are justified;
for, whom he justified, them he also glorified, Rom. viii. 30. But this
is not the privilege of all; therefore they who are not justified, or
whose sins are not pardoned, are the subjects of the punishment of
sin in the world to come. This is a very awful subject, and should be
duly improved, to awaken our fears, and put us upon using those
means, which God has ordained to escape it. But I shall not, in this
place, enlarge upon it, since it is particularly insisted on under
another answer,[85] and therefore I shall only observe, that, as sin is
objectively infinite, as being against an infinite God, it deserves
eternal punishment. And therefore all the punishments inflicted on
sinners, in this world are not proportioned to it; and consequently
there are vials of wrath, reserved in store, to be poured on those,
who wilfully and obstinately persist in their rebellion against God,
and the punishment will be agreeable to the nature of the crime; so
that as sin is a separation of the heart and affections from God, and
contains in it a disinclination to converse with him, as well as
unmeetness for it, the punishment thereof will consist in a
separation from his comfortable presence, and that is to be
separated from the fountain of blessedness, which must render the
soul beyond expression, miserable. This is generally called a
punishment of loss; and there is besides it, a punishment of sense,
expressed by those grievous torments, which are to be endured in
soul and body; the soul, in a moral sense, may be said to be capable
of pain, as it has an afflictive sensation of those miseries which it
endures; and the body is so in a natural sense, which, as it has been
a partner with the soul in sinning, must likewise be so in suffering.
And this farther appears inasmuch as the body endures several pains
and evils, as punishments of sin in this life, which shall be continued,
and increased in another. This is usually expressed by that
punishment, which is most terrible, namely, of fire; and the place in
which it is inflicted, is hell, and the duration thereof is to eternity.
But of these things elsewhere.[86]
 Quest. XXX.

Quest. XXX. Doth God leave all mankind to perish in the state of
sin and misery?
Answ. God will not leave all mankind to perish in the state of sin
and misery, into which they fell by the breach of the first
covenant, commonly called, the covenant of works; but of his
mere love and mercy, delivereth his elect out of it, and bringeth
them to an estate of salvation by the second covenant,
commonly called the covenant of grace.

Hitherto we have considered man as made upright, and having

many blessings in possession, and more in expectation, according to
the tenor of the covenant he was under. We have also observed the
first entrance of sin into the world, with all those miseries that
attended it; and we are now led to speak of that inestimable display
of divine love and grace, which appears in our salvation, which is
considered more generally in this answer; wherein there is,
I. Something supposed, namely, that if God had left man in the state
into which he brought himself by sin, he would have perished for
ever. He was not only in danger of ruin and destruction, but sunk
into it. He was like a brand in the fire, that would soon have been
consumed, had he not been plucked out of it. His state was not only
miserable, but hopeless, inasmuch as he could not think of any
expedient how he might recover himself. He was guilty, and no
creature could make atonement for him; separated from the
comfortable presence of God, whose terrors made him afraid, and
whose hand was heavy upon him; neither could he apply himself to
any one, who would interpose or appear in his behalf, whereby he
might be restored to the enjoyment of those privileges, which he
had forfeited and lost. What tongue can express, or heart be suitably
affected with the misery of this condition! And this would have been
our deplorable case for ever, had we been left of God in our fallen
state. But we have, in the gospel, a door of salvation opened, or
glad tidings proclaimed therein, to those who were sunk as low as
hell, which is the only spring and hope of comfort, to those who are
afflicted with a sense of their sin and misery. Accordingly, it is farther
observed,
II. That God will not leave all mankind to perish in that state, but
designed to deliver his elect out of it, and bring them into a state of
salvation. That God designed not to leave mankind in this miserable
condition, appears from the discovery he has made of the way of
salvation which was contained in that promise, which God gave to
our first parents, respecting the seed of the woman, who was to
break the serpent’s head; or the Saviour’s being manifested that he
might destroy the works of the devil; and all the promises contained
in the gospel, are, as it were, a farther improvement on it, or a
continued declaration of God’s purpose relating to the salvation of
his people. The work of redemption wrought out by Christ, as God
incarnate, was a wonderful discovery of this great truth, that God
had a design to recover and save lost sinners; and all the gifts and
graces of the Spirit, by whom the redemption purchased by Christ, is
applied, and that joy and peace, which they have in believing, which
are, as it were, the first fruits of eternal life, these are all a
convincing proof that God determined not to leave man to perish in
his fallen state. And to this we may add, that even the malice and
rage of Satan, and all the endeavours used by him, to defeat this
design, and the glorious victory which God enables his people to
obtain over him, who are made more than conquerors through him
that loved them; these are so many convincing proofs, that God
designed not to leave man, in his ruined condition, but to make
known to him the way of salvation; first, to make him meet for it,
and then to bring him to the possession of it.
Salvation is an inestimable privilege, containing in it all the
ingredients of blessedness, such as are adapted to the condition of
miserable sinners; and it is a very comprehensive one; which will
appear, if we consider what we are hereby delivered from, and what
we are possessed of. There is a great variety of blessings contained
in the former of these; as, we are saved from sin, namely, from the
guilt thereof in justification, and from the dominion thereof in
sanctification, and from that bondage we were liable to, whereby we
were in perpetual dread of the wrath of God, desiring to fly from his
presence, and naturally inclined to yield ourselves subjects and
slaves to his greatest enemy: all these we are delivered from. And
there are many positive blessings and privileges, which we are made
partakers of; such as, grace and peace begun here, and perfected in
glory hereafter; and these are not only such as exceed our highest
desert, but tend to make us completely and eternally happy. Here
we are to consider,
1. The subjects of this privilege. Salvation is not extended to all
miserable creatures; for, fallen angels, who were the first that
rebelled against God, were left to perish, without hope of salvation,
being reserved for ever in chains under darkness. And as for fallen
man, how extensive soever the proclamation of salvation in the
gospel is, as it is now preached to all nations, and all who sit under
the sound thereof, are commanded and encouraged to press after it;
yet this privilege is applied only to those who were ordained to
eternal life. The purpose of God, relating hereunto, and the
application thereof, are joined together in that golden chain of
salvation, Whom he predestinated, them he also called; and whom
he called, them he also justified; and whom he justified, them he
also glorified, Rom. viii. 30. But this has been more particularly
considered elsewhere[87].
2. Here is the only moving cause, or reason, why God bestows this
great salvation, or why he has designed to bring any of the sons of
men to it; and that is his mere love and mercy. Salvation, whether
considered in its first rise, in God’s eternal purpose, or in the
execution thereof in the work of conversion and sanctification, as
well as in the completing of it in glorification, is ascribed to the
sovereign grace and mercy of God. Are we Chosen in Christ to be
holy, or predestinated to the adoption of children by him? this is said
to be to the praise of the glory of his grace, Eph. i. 4-6. And the
apostle elsewhere, when resolving this great privilege of salvation, in
all the branches of it, namely, regeneration, renovation, and
justification, into the same original cause and ground thereof, to wit,
the kindness, love, and grace of God, excludes all those works of
righteousness which we have done, from being the inducement, or
moving cause leading to it, Tit. iii. 4-7. so that it was the grace of
God that laid the foundation stone, and it is that that brings the
work to perfection.
To make this farther appear, let it be considered, that salvation must
either be of grace, or of debt; either the result of God’s free favour
to us, or it must proceed from some obligation, which he is laid
under by us, to confer this privilege upon us. Now it is certain, that it
cannot take its rise from any obligation that we can lay on him; for
whatever difference there is between the best of saints and the
worst of sinners, it is from God, and not from the sinner himself. We
have nothing but what we first received from him, of whom, and
through whom, and to whom are all things, Rom. xi. 35, 36.
Moreover, this salvation must be conferred, in such a way, as
redounds to the glory of him, who is the author of it, whereby all the
boasting in the creature is excluded, and therefore it cannot take its
rise from any thing done by us; it is not of works, lest any man
should boast, Eph. ii. 9. And, indeed, this is contrary to the main
design of the gospel, which is, that no flesh should glory in his
presence. And the circumstances in which those are, who are said to
be the objects of salvation, are such as argue it to be altogether of
grace; for, whom did the Son of Man come to seek and to save, but
them that were lost? or, to whom was the way of salvation
discovered, but to those who were going astray from God, and were
neither inclined to return to him, nor apply themselves to any one,
who might direct them how to regain his lost favour? And, if they
had, it would have been to no purpose; since no creature could
make known the way of salvation, any more than apply the blessings
contained therein.
Were man only to be considered as a creature, and so not properly
the object of salvation, which is no other than a lost sinner; or did
he expect nothing else but some effects of common goodness, or
the blessings of nature, he could not expect them in a way of merit;
for that is contrary to the dependance of the creature on God;
therefore the blessings of Providence must be considered as the
result of his free favour. And were man in a sinless state, and able to
perform perfect obedience, as he was at first, his ability hereunto
must be supposed to be an unmerited favour; and accordingly the
obedience performed would be no other than a just debt due to
God, and therefore would afford him no plea, from any merit of
condignity, for the conferring any privilege, as a reward thereof: this
therefore, must be the result of the divine favour.
But, when we consider him as a sinner, he is altogether unable to do
what is good; and therefore, if salvation were entirely to depend on
our performing obedience, so that any failure therein would deprive
us of it, we should never attain it; for this obedience would be so
imperfect, that God could not, in honour, accept of it. But alas! fallen
man is so far from any disposition, or inclination to perform
obedience, that his heart is naturally averse to it; The carnal mind is
enmity against God; for it is not subject to the law of God, neither
indeed can be, Rom. viii. 7. If therefore, such an one is saved, and
that in such a way, that God is pleased to love him, and manifest
himself to him, it must be a wonderful instance of divine grace,
which no one, who has experienced it, can think on, but with
admiration, especially when considering how discriminating it is; as
one of Christ’s disciples said unto him, How is it that thou wilt
manifest thyself unto us, and not unto the world? John xiv. 22.
3. Having considered salvation, as designed for all the elect, we
proceed to consider the means of their attaining it; or their being
brought into a state of salvation by the second covenant, commonly
called the covenant of grace. As salvation is ascribed to the grace of
God; so it is an instance of condescending goodness, that our faith,
relating hereunto, should be confirmed by such a dispensation, as is
generally styled a covenant. Thus David, speaking concerning it,
says, He hath made with me an everlasting covenant, ordered in all
things, and sure; for this is all my salvation, and all my desire, 2
Sam. xxiii. 5. This covenant, as to what respects the parties
concerned therein, and the manner in which the grace of God is
displayed in it, together with the various dispensations, or
administrations thereof, is particularly considered under the five
following answers. The only thing, that remains to be insisted on in
this, is its being called the Second Covenant, as opposed to the
covenant of works, which is styled the First. The covenant of works
has been considered under a foregoing answer[88]; and therefore all
that I shall observe, concerning it, at present, is, that though life
was promised therein, as including all those blessings, which were
suited to the state of man in innocency, yet there was no promise of
salvation in it, which is the restoring of forfeited blessings, or a
recovery from a state of death and ruin. In this respect, the
covenant of grace is opposed to it.
Again, though Adam was the head of that covenant, whose
obedience, or apostacy, would convey life or death to all his
posterity, whom he represented, yet he stood not in the relation of a
Mediator, or surety, to them, for that was inconsistent with the
dispensation he was under, and is applicable to no other covenant,
than that which we are considering, as thus opposed to it.
Moreover, perfect obedience was demanded, as a condition of man’s
attaining life, and this he was thoroughly furnished to perform;
whereas, in the covenant of grace, if God should insist on our
performing perfect obedience, the condition would be in its own
nature impossible, and therefore we should hereby rather be
excluded from, than brought into a state of salvation; and whatever
obedience we are engaged to perform, as expectants of salvation,
this is entirely owing to the grace of God, by which we are what we
are, as well as attain to the blessings we hope for: Herein the
covenant of works, and the covenant of grace, differ.
The next thing that we are to observe, is, that the covenant of grace
is called the Second Covenant; and this leads us to enquire, whether
we have any ground, from scripture, to conclude, that there are
more covenants than these two; or, at least, whether what we call
the Second Covenant, or the covenant of grace, may not be
subdivided into two covenants; since the apostle seems to speak of
two covenants made with fallen man, viz. one that was made with
the Israelites, given from mount Sinai, which was designed to
continue no longer than that dispensation they were under, lasted;
and the other is, that which the church has been under, ever since
the gospel dispensation was erected, which is to continue to the end
of the world. These are described by their respective properties, in
an allegorical way, and illustrated by a similitude, taken from two
mountains, Sinai and Sion; and two persons, mentioned in scripture,
Agar and Sarah: The former of these is said to gender unto
bondage; the latter brings those, who are under it into a state of
liberty, Gal. iv. 24. & seq. and one of these covenants is said to be
better than the other, and particularly called a new covenant; the
other is represented as decaying, waxing old, and ready to vanish
away, Heb. viii. 6, 8, 13.
Moreover, the apostle seems to speak of more covenants than one,
made with the Jewish church; for he says, that to them pertaineth
the adoption, and the glory, and the covenants, Rom. ix. 4. &c. and
elsewhere, speaking concerning the Gentiles, as aliens from the
commonwealth of Israel, he adds, that they were also strangers
from the covenants of promise, Eph. ii. 12. which seems to argue,
that there were more than two covenants with man; one with
innocent man; the other, the gospel-covenant, which we are under;
and, besides these, there were other covenants, made with Israel,
which seems to carry in it the appearance of an objection, to what
was before observed, that there was, in reality, but two covenants,
and that whenever we read of any covenant in scripture, it is
reducible to one of them.
This may, without much difficulty, be accounted for, consistently
therewith, if we consider the sense of those scriptures above
mentioned.
First, As to those scriptures, that seem to speak of two distinct
covenants, made with fallen man, to wit, one with the Israelites, the
other, that which we are under, they really intend nothing more than
two different dispensations of the covenant of grace; in which sense
we are to understand the apostle, when he speaks of the two
covenants, the Old and the New, the First and the Second: the
covenant is the same, though the dispensation of the grace of God
therein, or the way of revealing it to men, differs. But this will be
more particularly insisted on in those following answers, which
respect the various administrations of grace, under the Old and New
Testament; therefore we proceed,
Secondly, To enquire into the meaning of those other scriptures,
before-mentioned, which seem to speak of more covenants than
one, which the Jewish nation was under. By the covenants there
mentioned, the apostle seems to refer to some different times, or
periods of the church, before our Saviour’s incarnation, of which
some divines take notice of four; in each of which, there was
something new and distinct from the rest, in the dispensation of
divine providence towards the church. The first of these took its rise
from the promise which God gave to man, as soon as he fell,
relating to that salvation, which was to be brought about, in its
proper time, by the seed of the woman. The second period of the
church began after the flood, when God is said to have revealed his
covenant to Noah, which he established between him and all flesh
upon the earth, Gen. ix. 17. A third remarkable period, or change of
affairs in the church, was, when God called Abraham out of an
idolatrous country, to sojourn in the land of promise, as in a strange
country, at which time he established his covenant with him,
promising to be a God to him, and his seed, and instituting
circumcision as a token thereof, Gen. xvii. 7-11. upon which
occasion, this particular dispensation thereof is called, The covenant
of circumcision, Acts vii. 8. The fourth and last dispensation, or
period, which more especially respected the seed of Abraham, as
increased to a great nation, is what we read of, soon after they were
delivered from the Egyptian bondage, when God was pleased to
separate that nation, as a peculiar people to himself, and sent Moses
from mount Sinai, where he appeared to them, to demand their
explicit consent to be his people; upon which occasion, when they
had promised, that all that the Lord had said, they would do and be
obedient, and a public and solemn sacrifice was offered, and the
people sprinkled with the blood thereof, it is said, They saw God,
and did eat and drink, as a farther sign and ratification of this
dispensation of the covenant, Exod. xxiv. 1-11. and afterwards many
statutes and ordinances were given them, containing those laws,
which God required of them, as a covenant people; and this
continued till the gospel-dispensation, which succeeded it, was
erected. This seems to be the meaning of what the apostle speaks,
in the scriptures before cited, when he says, that the church of the
Jews had the covenants, as intending nothing else thereby, but the
dispensation of the covenant of grace, as subdivided into several
periods, during the various ages of the church, from the fall of Adam
to our recovery by Christ. Therefore, though those dispensations
were various, yet whatever God has transacted with man, in a
federal way, may be considered under two general heads; the first
called the covenant of works; the other, the covenant of grace; the
latter of which is to be farther considered, under the following
answers.
 Quest. XXXI.

Quest. XXXI. With whom was the covenant of grace made?

Answ. The covenant of grace was made with Christ as the second
Adam; and in him, with all the elect, as his seed.

As the covenant of grace is opposed to that which was made with

Adam, as the head of mankind, so it is considered in this answer, as
made with the second Adam, and, in him, with all his elect, who are
described, by the Psalmist, as a seed that should serve him, which
should be accounted to the Lord for a generation, Psal. xxii. 30. and
the prophet Isaiah, speaking of them, says, He shall see his seed,
Isa. liii. 10. In explaining this answer, we shall consider,
I. What we are to understand by a covenant in general, and more
particularly how it is to be understood, as used in scripture. The
word commonly used in the Old Testament,[89] to signify a covenant,
being taken in several senses, may be better understood, by the
application thereof, in those places, where we find it, than by
enquiring into the sense of the root, from whence it is derived.
Sometimes, indeed, it signifies such a compact between two parties,
as agrees with our common acceptation of the word, especially
when applied to transactions between man and man; as in the
covenant between Abraham, and those neighbouring princes, that
were confederate with him, where the same word is used, in Gen.
xiv. 13, and in the covenant between Isaac and Abimelech,
mentioned in Gen. xxvi. 28, 29. and in that between Jonathan and
David, in 1 Sam. xx. 16, 17. in all which instances there was mutual
stipulation, and re-stipulation, as there is in human covenants; and,
for this reason, some apply those ideas to the word, when it is used
to signify God’s entering into covenant with man.
But there is another acceptation thereof when God is represented as
making a covenant with man which is more agreeable to the divine
perfections, and that infinite distance there is between him and us;
therefore we find in several places of scripture, that when God is
said to make a covenant there is an intimation of some blessings
which he would bestow upon his people, without any idea of
stipulation, or re-stipulation, annexed to it: thus we read, in Jer.
xxxiii. 20. of God’s covenant of the day and night, or that there
should be day and night in their season; and, in Gen. xi. 9, 10, 11.
of God’s establishing his covenant with Noah, and his seed, and
every living creature, that all flesh should not be cut off any more,
by the waters of a flood. And, in Ezek. xxxiv. 25. when God promises
to cause evil beasts to cease out of the land, and that his people
should dwell safely in the wilderness, and that he would confer
several other blessings upon them, mentioned in the following
verses; this is called, his making with them a covenant of peace.
And, when God promises spiritual blessings to his people, in Isa. lix.
21. he says, This is my covenant with them; my Spirit that is upon
thee, and the words that I have put into thy mouth, shall not depart
out of thy mouth, nor out of the mouth of thy seed, nor out of the
mouth of thy seed’s seed, saith the Lord, from henceforth, and for
ever.
Moreover, sometimes the Hebrew word, which we translate
covenant, is used to signify a statute, or ordinance, which God has
established, or appointed, in his church: thus, in Numb. xviii. 19.
when God ordained, that Aaron and his sons should have the heave-
offerings of the holy things, he says, These have I given thee, and
thy sons and thy daughters with thee, to be a statute for ever, and
adds, in the words immediately following, It is a covenant of salt for
ever, before the Lord.
And as for the word used in the New Testament,[90] by which the LXX
generally translate the Hebrew word, before-mentioned, in the Old
Testament, this signifies the same thing; so that both the words
imply little more than a divine establishment or ordinance, in which
God gives his people ground to expect promised blessings, in such a
way, as redounds most to his own glory; and at the same time, they,
who are expectants thereof, are not exempted from an obligation to
perform those duties, which this grace obliges them to, and which
will be an evidence of their right to them.
And I cannot but farther observe, that among other acceptations of
the word, especially as used by the apostle, in his epistle to the
Hebrews, in chap. ix. 15-18. it signifies a Testament; which word
some who treat on this subject, rather choose to make use of, than
to call it a covenant, being warranted so to do, by the sense given of
it in this scripture; and their reason for it is, not only because, as the
apostle says, it was confirmed by the death of the Testator;[91] but
because they conclude, that this more conduces to the advancing
the grace of God, in this dispensation, than to style it a covenant, in
that sense, in which the word is commonly used, when applied to
other matters: but I would rather acquiesce in that medium, betwixt
both extremes, which some have given into, who join both the ideas
of a covenant and a testament together[92], and style it, in some
respects, a covenant, and, in others a testament. If it be called a
covenant, they abstract from the ideas thereof, some things, that are
contained in the sense of the word, as applied to human contracts,
and add to it other things, contained in a testament; such as the
giving or bequeathing certain legacies, as an act of favour, to those
who are denominated, from thence, legatees, interested in those
gifts that are thus disposed of by the will of the testator. Or if, on the
other hand, we call it a testament it seems very agreeable, to this
dispensation, to join with it the idea of a covenant, more especially
as to what contains the concern of Christ herein, as the Head
thereof, or the Person in whom all the benefits, contained in this
testament, are first reposed, as they are purchased by his blood,
and, as the consequence thereof, applied by his Spirit. And this
agrees very well with the subject-matter of this answer, in which the
covenant is said to be made with him, and with the elect in him, as
well as with what is contained in that answer immediately following,
in which the covenant of grace is described in such a way, as they
describe it, who say that it was made with believers. This is
necessary to be premised, that we may not, in our explication of this
doctrine, advance any thing which is inconsistent with its being a
covenant of grace: and, that we may farther consider this matter, we
shall proceed to shew,
II. What there is in the idea of a covenant, as we generally
understand the word, when applied to signify a contract between
man and man. In this case, there are two parties, one of which is
said to stipulate, or enter into a covenant with the other, in which he
makes a proposal, that he will confer some favours on him, upon
certain conditions, provided he will oblige himself to fulfil them; and
the other party complies with the proposal made, and, in
expectation of those advantages, consents to fulfil the conditions
enjoined, and accordingly is said to re-stipulate; as when a person
engages another to be his servant, and to give him a reward for his
service; and the other consents to serve him, in expectation of the
wages which he engages to give him: in this case, each party is
supposed to be possessed of something, which the other has no
right to, but by virtue of this contract made between them: thus the
servant has no right to the rewards, which his master promises, nor
has the master any right to his service, but by mutual consent. Each
party also proposes some advantage to himself, and therefore, when
they enter into this agreement, they are supposed, in some respects,
to stand on a level with each other. No one will enter into a covenant
with another, for the performing that which he had an antecedent
right to; nor will any one engage to perform any service, as a
condition of his receiving those benefits, which he had a right to,
without any such condition enjoined on him. Moreover, when two
parties are said to enter into covenant with one another, they are
supposed, in some respects, to stand in need of some things, which
they had before no right to; one party needs the reward proposed;
the other, the service which he enjoins, as a condition of his
bestowing it. These things are generally supposed, and contained in
contracts between man and man.
III. When God is said to enter into covenant with man, what method
soever we take to explain this federal transaction, we must take
heed that we do not include in it any thing that is inconsistent with
his infinite sovereignty, or argues him to be dependent on his
creatures, as though he had not an antecedent right to their
obedience, which he demands in this covenant, or it were left to
man’s arbitrary will whether he would perform it or no. Though men
may be said to have some things in their own power, so that one has
a right to that, which another has no right to, but by his own
consent, and are entirely left to their liberty, whither they will
consign over that right, which they had to it, to another, who could
not otherwise lay claim to it; yet this is by no means to be applied to
man when considered as having to do with the great God. The best
of creatures have no right to any thing, separate from his arbitrary
will; and therefore though stipulation and re-stipulation are proper
words, when applied to a man’s covenant, they ought not to be
made use of, when we explain this covenant between God and man.
IV. Though the parties concerned in the covenant, as explained in
this answer, to wit, God the Father, and Christ the Head of his elect,
are both divine Persons, so that one of them is not infinitely below
the other, as man is below God; and therefore it is more properly
called a covenant, in this respect, than that which God is said to
enter into with man, (and, if stipulation and re-stipulation is, in any
respect, applicable to the divine dispensation, it may be applied in
this case:) nevertheless, there are some things, which are implied in
the idea of a covenant between man and man, that cannot,
consistently with the glory of these divine Persons, be contained in
this federal transaction between them; particularly, as he that enters
into covenant with another, proposes some advantage to himself
hereby: thus a master, when he stipulates with one to be his
servant, is supposed as much to need his service, as the servant
does the wages that he promises to give him; there is a kind of
mutual advantage arising from thence: but, in the covenant of grace,
whether God be said to make it with man, or with Christ, as the
Head of his elect, the advantage that arises from thence is our’s, and
not God’s. In this respect, what was done by Christ, made no
addition to the essential glory of God, or the divine blessedness, any
more than man can be said, in that respect, to be profitable to him:
thus some understand those words of the Psalmist, as spoken by our
Saviour, when he says, My goodness extendeth not to thee, but to
the saints which are in the earth, Psal. xvi. 2, 3. and this agrees very
well with some other things, contained in the same Psalm, which are
expressly, in other parts of scripture, applied to him; and, if so, then
the meaning is, that whatever glory God the Father designed to
demonstrate by this federal transaction with his Son; yet he did not,
as men do, by entering into covenant with one another, propose to
receive any addition of glory from it, as though he were really to be
profited thereby.
Again, when men enter into covenant with one another, they are
supposed to have different wills, and accordingly they might refuse
to enter into those engagements, which they bring themselves
under, as well as comply with them; the obligation, on both sides, is
founded in mutual consent, and that is supposed to be arbitrary:
but, when we consider the eternal compact between the Father and
the Son, we must conclude, that though they be distinct as to their
personality, yet, having the same essential perfections, the will of
the Father and the Son, cannot but be the same. Therefore when
many, who explain this doctrine, represent one as proposing, the
other as complying, with the proposal; one demanding, the other
expecting, and each depending on mutual promises, made by one to
the other, this, it is true, seems to be founded on some scripture-
expressions to the same purpose, wherein the Holy Ghost is pleased
to condescend to make use of such modes of speaking, which are
agreeable to the nature of human covenants, as he does in various
other instances; nevertheless, we must not so far strain the sense of
words, as to infer, from hence, any thing that is inconsistent with the
divine glory of the Father and the Son. And to this we may add, that
no act of obedience can be performed by a divine Person, in the
same nature, as there cannot be an act of subjection in that nature,
which is properly divine; and consequently when we consider Christ,
in this respect, as entering into covenant, and engaging to perform
those conditions, which were insisted on therein, these are supposed
to be performed by him, as Mediator, or God incarnate, in his human
nature; and, in this respect, he is the Head of the covenant, which is
made with him, and, in him, with the elect. Therefore we must
suppose, when we speak of a covenant between the Father and the
Son, that, whatever be the will of the Father, the same is the Son’s
will; and whatever conditions the Son consented to perform, as
stipulated in this covenant, it was in his human nature that the work
was to be done; and therefore it is well observed, in some following
answers, that he, who is the Head or Mediator of this covenant, is,
as it was absolutely necessary for him to be, both God and man, in
one Person. But of this more hereafter.
V. There are several expressions used, in scripture, that give us
sufficient ground to conclude, that there was an eternal transaction
between the Father and the Son, relating to the salvation of his
elect, which, if explained agreeably to the divine perfections, and
consistently with the glory of each of these divine Persons, is not
only an undoubted truth, but a very important article of faith, as it is
the foundation of all those blessings, which are promised, and
applied to us in the covenant of grace, in which is all our salvation
and our hope. Here let it be considered, that, when we speak
concerning a covenant, as passing between the Father and the Son,
we understand thereby, that there was a mutual consent between
them both, that the work of our redemption should be brought
about in such a way, as it was, by our Saviour, when this eternal
agreement had its accomplishment; and accordingly the Father is
said to have set him up, as the Head of his elect, from everlasting,
Prov. viii. 23. and ordained, that he should execute those offices,
which he was to perform, as Mediator, and receive that revenue of
glory, that was the result thereof; and the Son, as having the same
divine will, could not but consent to do this; and this is called, his
eternal undertaking; and, both these together, are styled the eternal
covenant, between the Father and him.
For the proof of this doctrine, we might refer to those several
scriptures that speak of our Saviour as called, and given for a
covenant of the people, Isa. xlii. 6. and fore-ordained, 1 Pet. i. 20. to
perform the work which he engaged in, in the behalf of his elect;
and also consider him as consenting to do every thing for his people,
which he did in time, and to stand in every relation to them, that
was subservient to their redemption and salvation, which he could
not but do, as having the same divine will with the Father; and
without his consent, it could not properly be said that there was a
covenant between them. We might also prove it from those several
scriptures, that speak of him, as sanctified and sent into the world,
John x. 36. to act as Mediator, sealed by the Father, John vi. 27. and
receiving a power to lay down his life, and take it up again, John x.
18. that so he might answer the great end of our redemption
thereby; and also, from his being empowered to execute the offices
of a Prophet, Priest, and King; confirmed in his priestly office by the
oath, Psal. cx. 4. Heb. vii. 21. of the Father, sent by him to execute
his Prophetical office to those whom he was to guide in the way of
salvation; and, as God’s King, set on his holy hill of Zion, Psal. ii. 6.
When we consider all these things done, on the Father’s part, as
antecedent to Christ’s acting as Mediator, and, at the same time,
when we compare them with other scriptures, that speak of the Son,
as consenting to do the will of God, or complying with his call, willing
to be and do whatever was necessary, to secure the great ends
designed thereby; when we consider him, as taking the human
nature into union with the divine, not without his own consent
thereunto, and as bearing the punishment due to our sin, which it
would not have been just for God to have inflicted, without his will
or consent; I say, this mutual consent between the Father and the
Son, that those things should be done which were subservient to the
redemption and salvation of the elect, which the scripture is very
express in giving an account of, these are a sufficient foundation for
our asserting, that there was a covenant between the Father and the
Son relating thereunto.
But now we shall enquire, more particularly, into the sense of those
scriptures, on which this doctrine is founded. And here we cannot
wholly pass over what we read, in Psal. cxix. 122. Be surety for thy
servant for good; and Hezekiah’s prayer, in Isa. xxxviii. 14. I am
oppressed; undertake, or be surety, for me. The Hebrew words are
the same in both places, and signifies, not barely to confer some
privileges on persons, but to do this under the character of a surety;
and therefore when David and Hezekiah pray that they may be
delivered, either from their enemies, or their afflictions, by
addressing themselves to their Deliverer under this character, it must
be supposed that they understand him, as having undertaken to be
a Surety for his people, which is a character that belongs only to the
Son. And since it is so evident, that his Mediatorial work and
character was so well known to the Old Testament church, as their
salvation was equally concerned herein with ours; and, since they
are often represented as addressing themselves to him by faith and
prayer, it seems more than probable that he is so considered in
these texts, when it is desired that he would be surety for them,
namely, that as he was appointed by the Father, and had
undertaken, by his own consent, to stand in that relation, they pray
that they might be made partakers of the benefits arising from
thence.
There is also another scripture, in which the same word[93] is used,
which seems to be applied to our Saviour, viz. in Jer. xxx. 21. Their
nobles, or, as it ought to be rendered, in the singular number, their
noble, or magnificent person, shall be of themselves, and their
governor shall proceed from the midst of them; and I will cause him
to draw near, and he shall approach unto me; for who is this that
engaged his heart to approach to me, saith the Lord? This sense of
the text is very agreeable to several other prophecies, relating to the
Messiah’s being of the seed of Israel; and when it is said, I will cause
him to draw near, and he shall approach unto me, it implies, that he
should sustain the character, and perform the work of a surety, in
the behalf of his people, for that is the proper sense of the word
there used; for who is this that hath engaged his heart unto me?
that is, who is there, among the sons of men, that dares engage in
this work, or is qualified for it? Or it may be understood with a note
of admiration; that is, how glorious a person is this, who hath
engaged his heart, or (as it was determined that he should) has
freely consented to approach unto me, that is, in so doing, to act as
a surety with me for my people! And that this is a more probable
sense of the text, than to suppose that it is meant either of
Zerubbabel, or some other governor, that should be set over them,
after the captivity, appears, if we compare it with ver. 9. in which it is
said, They shall serve the Lord their God, and David their king,
which can be meant of none but Christ, inasmuch as David was
dead; and none that sat on his throne, or descended from him, can
be called David in this place, because divine worship is said to be
performed to him, which could not be done without idolatry, which
no true sense of scripture can give countenance to; and this is a
character given of our Saviour in other scriptures: thus, in Ezek.
xxxiv. 24. I will be their God, and my servant David a Prince among
them; and, in Hos. iii. 5. They shall seek the Lord their God, and
David their King, and fear the Lord and his goodness in the latter
day; that is, they shall adhere, and give divine worship, to the
Messiah, whom their fathers rejected, when they are converted, in
the latter days. Now it is this David, their King, who is said to have
engaged his heart to approach unto God; and then, in the words
immediately following, ver. 22. God reveals himself, as a covenant-
God, to them, which is the consequence of Christ’s engaging his
heart to approach unto him: Ye shall be my people, and I will be
your God. Now this proves an eternal transaction between the Father
and the Son, in that the Father wills, or determines, that he shall
draw near, or approach to him, as a surety, and the Son consents, in
that he has engaged his heart to do it; and all this with a design that
his covenant should be established, and that he should be a God to
his people.
There is another scripture which proves that there was a federal
transaction between the Father and the Son, from several
expressions therein used, namely, in Isa. xlii. 1, 6. which is, beyond
dispute, spoken concerning our Saviour; for it is applied to him in
the New Testament, Matt. xi. 18-21. Herein God the Father calls him
his Servant, as denoting that it was his will, or (to use that mode of
speaking, which is generally applied to covenants between man and
man) that he stipulated with him, to perform the work which he
engaged in, as Mediator, to which he is said to be called in
righteousness; and, with respect to his human nature, in which he
performed it, he is styled God’s elect, as fore-ordained hereunto, and
the person in whom his soul delighteth, as he is glorified by him in
the faithful discharge thereof; and, that he might not fail therein,
God promises to hold his hand, and keep him; and, as the result of
his having accomplished it, to give him for a covenant of the people,
for a light of the Gentiles.
And elsewhere, in Isa. xlix. 8, 9. which also appears to be spoken to
Christ, not only from the context, but from the reference to it in the
New Testament, 2 Cor. vi. 2. In an acceptable time have I heard
thee, and in a day of salvation have I helped thee; and I will
preserve thee, and give thee for a covenant of the people, to
establish the earth, to cause to inherit the desolate heritages; that
thou mayest say to the prisoners, Go forth; to them that are in
darkness, Shew yourselves, we have a plain intimation of his being
ordained by the Father to perform that work, which he was engaged
in, as Mediator; and his being given for a covenant of the people,
signifies his being sent into the world, in pursuance of a covenant, in
which the salvation of his people was contained. And there is
another scripture, in which our Saviour, speaking to his disciples,
says, in Luke xxii. 29. I appoint unto you a kingdom, as my Father
hath appointed me;[94] or, I confer the blessings of this kingdom
upon you, in a covenant way, as my Father hath appointed me to do,
in that eternal covenant, which passed between him and me.
Again, there are several rewards, which were promised to him, as
the consequence of his discharging the work committed to him,
some of which respected that glory which belongs to his person, as
Mediator; and others, more especially, respected the salvation of his
people, and therein the success of his undertaking: thus it is said, in
Isa. liii. 10. When thou shalt make his soul an offering for sin, he
shall see his seed; he shall prolong his days, and the pleasure of the
Lord shall prosper in his hands; together with several other things
relating to the event, and consequence of his performing the work
he was engaged in.
Moreover, as he was called to this work, or, as it was, as we before
explained it, the result of the Father’s will, that he should perform it;
so we have elsewhere an account of his own consent, as implying,
that it was the result of his own will, as well as his Father’s: thus it is
said, in Psal. xl. 6-8. Mine ears hast thou opened, or bored: alluding
to a custom used under the ceremonial law, by which the willing
servant was signified to be obliged, by his own consent, to serve his
master for ever, Exod. xxi. 5, 6. Thus God the Father, engaged
Christ, if I may so express it, to perform the work of a Mediator; and
then we have an account of his consent hereunto, when he says, Lo,
I come, I delight to do thy will, O my God; yea, thy law is within my
heart; and this mutual consent is farther expressed in Isa. l. 5. The
Lord God hath opened mine ear, and I was not rebellious; neither
turned away back.
And he is farther represented, as making a demand, or insisting on
the accomplishment of what was stipulated in this covenant; and
this he had a warrant to do from the Father, in Psal. ii. 8. Ask of me,
and I shall give thee the heathen for thine inheritance, and the
uttermost parts of the earth for thy possession. These, and many
other scriptures of the like nature, sufficiently prove this doctrine,
that there was an eternal covenant between the Father and the Son,
relating to the redemption and salvation of the elect; and this implies
more than his being barely fore-ordained to perform the work he
was engaged in, as he is said to have been, 1 Pet. i. 2. for that,
alone, would not have proved that there was a federal transaction
between the Father and him; since it may be said of any one, who is
engaged in works of an inferior nature, that God, who called him to