76 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-24, NO.

1, JANUARY 1978

Efficient Maximum Likelihood Decoding of Linear Block Codes

Using a Trellis
JACK K. WOLF, FELLOW, IEEE

Abstract-It is shown that soft decision maximum likelihood Some, if not all, of these results can be deduced from
decoding of any (n,k) linear block code over GF(q) can be accom-
plished using the Viterbi algorithm applied to a trellis with no more
previously published papers. However, the practical sig-
than q (n-k) states. For cyclic codes, the trellis is periodic. When thisnificance of our technique makes it appear worthwhile to
technique is applied to the decoding of product codes, the number present them together here. A comparison between the
of states in the trellis can be muchfewer than q n-k. For a binary decoding complexity of our technique and that of the usual
(n,rz - 1) single parity check code, the Viterbi algorithm is equiv- word correlation decoding should amplify this point.
alent to the Wagner decoding algorithm.
Consider the maximum likelihood decoding of a (31,26)
binary Hamming code using channel measurement in-
I. INTRODUCTION formation. If word correlation decoding is utilized, the
WO DISTINCT error-control techniques exist received data would be compared with each of the 2zscode
T L for the reliable transmission of digital data over noisy words. Using the Viterbi algorithm, a trellis with only 25
communications channels: block codes and convolutional states is utilized. Both the word correlation decoder and
codes.There are many similarities and differences between the Viterbi decoder will decode to the same codeword and
these techniques. One important difference is that much thus give identical performance. In this case,the advantage
more efficient algorithms have been found for using of instrumenting a Viterbi decoder rather than a word-
channel measurement information (i.e., soft decisions) in correlation decoder should be obvious.
the decoding of convolutional codes than in the decoding The concepts presented in this paper have some simi-
of block codes. larity to the work of Bahl et al. [3] and that of Hartmann
This paper is concerned with the maximum likelihood and Rudolph [4]. However, in both of these papers, the
decoding of linear block codes using channel measure- authors were concerned with minimizing the probability
ment information. By maximum likelihood decoding, we of symbol error rather than the probability of word error.
mean a decoding algorithm which results in the minimum Miyakawa and Kaneko [5] have presented a different de-
probability of decoding to an incorrect code word when the coding algorithm for maximum likelihood decoding of
a priori probabilities of all the code words are equal. By linear codes using channel measurement information.
using channel measurement information, we mean that Their algorithm appears to require a decoder with greater
the decoding algorithm can utilize real numbers (e.g., the complexity than that discussed here. For example, for the
analog outputs of filters matched to the signals) associated (31,26) binary Hamming code, their decoder considers
with each component of the code word. The decoding al- 3684 error patterns. Chase [6] and others [7]-[lo] have
gorithm will be of particular use in decoding high-rate given suboptimum decoding algorithms which are rela-
codes, since the complexity of the algorithm will be tively simple to instrument, but which do not always
upper-bounded by a function of the number of parity achieve maximum likelihood decoding.
symbols. The technique described here has been applied to a
The following results are demonstrated in this paper. concatenated coding scheme where constant weight binary
1) Soft decision, maximum likelihood decoding of any block codes are transmitted over a fading channel. The
(n,k) linear block code over GF(q) can be accomplished details will be discussed elsewhere [ 111.
using the Viterbi [l] algorithm applied to a trellis having We first give a general formulation which holds for all
no more than qcnmk)states. linear block codes. We then consider the case of cyclic
2) If the linear code is cyclic, the trellis is periodic. linear codes. Finally, we consider product codes and show
3) If the linear code is a product code, the number of that, for such codes, the number of states in the trellis is
states required can be considerably less than qtnek). greatly reduced over what we might expect from the
4) For a binary (n,n - 1) single-parity check code, the treatment of the general problem.
Viterbi algorithm applied to the trellis is equivalent to
Wagner decoding [2]. II. LINEAR CODESOVER GF(q)
Denote the elements of the finite field GF(q) as ~j, j =
Manuscript received July 26,1976; revised April 29,1977. This research 0,1,2, * - * ,(q - 1). Consider alinear (n,k) code over G&‘(q)
was supported by the Air Force Office of Scientific Research, Air Force with parity check matrix H. Denote the ith column of H
Systems Command, USAF, under Grant AFOSR-74-2601. as hi, so that hi, i = 1,2, -. . ,n are (n - h)-tuples with el-
The author is with the Department of Electrical and Computer Engi-
neering, University of Massachusetts, Amherst, MA 01003. ements from GF(q). The codewords in the code are all the
001%9448/78/0100-0076$00.750 1978 IEEE
 WOLF: DECODING OF LINEAR BLOCK CODES 77

n-tuples Xwith elements from GF(q), such that HX= 0. K=O K=I K=2 K=3 K=4 K=5
Here 0 is the all-zero (n - k)-tuple.
We now define a trellis for this code. A trellis is a par-
ticular collection of nodes (or states) interconnected by
unidirectional edges.The nodes will be grouped into sets
indexed by a parameter k, k = 0,1,2, . . - ,n. A node indexed
by a particular value of k will be said to be at depth k.
Edges will be drawn between certain pairs of nodes at
depth k and at depth (k + l), for k = O ,l, * a. ,(n - l), with Fig. 1. Trellis for binary (5,3) code before expurgation.
the direction of the edge going from the node at depth k
to the node at depth (k + 1). At any depth k, there will be
at most qtn-“) nodes. The nodes at depth k will be iden- K=O K=l K=2 K=3 K=4 I(=5

tified by (n - k)-tuples, si(k), with elements from GF(q)

for certain values of i. All of the qcnBk)(n - k)-tuples are
assumed to be ordered from 0 to qcnPk)- 1, with 0 refer-
ring to the all zero (n - k)- tuple. si (k) is to be interpreted
as the ith (n - k)-tuple in this list. Since not all of the (n
- k)-tuples may correspond to nodes at a depth k, we let
Ik be the subset of the integers (O,l, . . . ,(qcn-IZ)- 1)) cor-
responding to those (n - k)-tuples which correspond to Fig. 2. Expurgated trellis for (5,3) code.
nodes at depth k. The edgesare labeled in a manner to be
described below. (5,3) code with parity check matrix
A trellis is a compact method of cataloging all of the 4 k 1 1 0 1 0
codewords of a linear code. Each distinct codeword cor- H= I = [hl h2 h3 h4 h.51.
respondsto a distinct path in the trellis. In order to seehow 1 0 1 0 1
this correspondence occurs, we describe how to construct Following the procedure outlined above, we find the trellis
the trellis for a particular code. before the expurgation of nodes to be as shown in Fig. 1
1) At depth k = 0, the trellis contains only one node, and, after expurgation of nodes, to be as shown in Fig. 2.
namely so(O),the all-zero (n - k)-tuple. For decoding, it is not necessaryto expurgate nodes in
2) For each k = O ,l, . . . ,(n - l), the collection of nodes the trellis, as the Viterbi algorithm can just as easily be
at depth (k + 1) is obtained from the collection of nodes used in the unexpurgated trellis.
at depth k by the formula
III. VITERBI DECODING USING A TRELLIS
sl(k + 1) = si(k) + +k+l,
Since the Viterbi algorithm is now a well-understood
for all i E Ik and j = O ,l, * * * ,(g - 1).
decoding procedure, only a brief description will be pre-
For each i in Ik, connecting lines are drawn between the sented here. We assume that decoding is to be accom-
node si(k) and 4 nodes formed from it at depth (k + 1) plished based upon the received n-tuple c with real com-
using the above formula. Each such line is labeled by ponents ci,cz, . . s c,. We assume no intersymbol inter-
the particular value of aj which formed sl(k + 1) from ferenceso that the jth component of c depends only on the
jth component of the transmitted code word, xi. We fur-
si(k).
3) We remove any nodes that do not have a path to the ther assumethat the noise contributions in each of these
all-zero state at depth n, and we remove all lines drawn to components are described by statistically independent
these expurgated nodes. random variables Ni, with probability density functions
There is a one-to-one correspondence between each fNi( ), i = 1,2,. . . ,n. Then the logarithm of the likelihood
codeword in the code and the sequence of aj on any path ratio of the data, given the transmitted codeword, is of the
from the all-zero node at depth 0 to the all-zero node at form
depth n. There are qk distinct paths through this trellis, log f ClX(Cl x) = & 1% ViVi(Ci Ixi)]
and each such path corresponds to a unique codeword.
For k = 0, we have only one state or node in the trellis: =5 Zi(Ci,Xi) ’ Z(X).
SO(O). For k = 1, we have q states: namely, ai( j = i=l

O ,l, * * * ,(q - 1). For an arbitrary depth k, 1 I k 5 n, we For a given received data sequence c, a maximum likeli-
have the states ajl’j,hi + aj,hz + * * - + ajkhk, where + is the hood decoderfinds the codeword X which gives the largest
addition operator defined for vectors with components value of Z(X). A brute force approach would suggesttrying
from the field GF(q). Note that the number of states at any all q k possible codewords.
depth cannot exceed qcnvk), the number of distinct (n - The Viterbi algorithm is a recursive algorithm whereby
k)-tuples with elements from GF(q). many codewords can be discarded from consideration in
We illustrate the construction of a trellis for the binary finding that X which maximizes Z(X). Referring to the
78 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. m24, NO. 1, JANUARY 19%

trellis introduced in the previous section, we can state the

procedure as follows. For each node si(k.) at depth k, assign
a real number V(si (k)) in accordance with the following
rules.
1) Fork = 0, set V(so(0)) = 0.
2) For all 1 E Ik+i, form V(sl(k + 1)) from V(si(k)) in
the following manner (k = 0,1,2, *. * (n - 1)):
Fig. 3. Encoder for cyclic code.
V(sl(k + 1)) = max [V(si(k)) i- Zk+l(Ck+l,aj)],
LyjG GF(q)
iEPk,lClk

where pk,J is the subset of Ik consisting of the set of indices

i such that, for some aj E GF(q), sl(k + 1) = si(k) +
ajhk+l.
3) Retain only that path to V(sl (k + 1)) from that si (k)
which gave the maximum in the above formula. 1
INPUT

4) At k = n, the sequenceof aj on the single remaining Fig. 4. Encoder for binary (15,ll) code withg(x) = x4 + x + 1.
path from the all-zero state at depth 0 to the all-zero state
at depth n corresponds to the codeword X which maxi-
mizes Z(X).
It should be noted that this algorithm can be used with
the unexpurgated trellis or the expurgated trellis. toll
0100
IV. DECODING OF CYCLIC CODES OVER GF(q) 0101
0110
011 I
For cyclic codes over GF(q), an alternative (but equiv- ,000
alent) method of forming the trellis is to associate the 1001
1010
nodes with the qcnmk)states of the (n - k) stage shift I01 I
,100
register used for encoding and decoding. For an (n,k) cyclic ,101
code over GF(q) with generator polynomial g(x) = go + glx I I IO
I I II
+ . *. + g,x: gi E GF(q), r = n - k, one form of the encoder
is as shown in Fig. 3. Fig. 5. Trellis for depths h = 0, 1, 2, 3, and 4.
The square boxes are storage devices for elements from
GF(q), the circles enclosing + signs are adders for elements
from GF(g), and the circles enclosing gi’s are multipliers 0000
0001
for elements from GF(q). (g,-i is the multiplicative in- 0010
verse of g,.) We enter the k messagedigits at the input with 001 I
oioo
the switches S1 and Sz in position 1. We then enter (n - 0101
k) O’sat the input with the switches Si and S’sin position 01 IO
01 I I
2. The output is the codeword. ,000
1001
The trellis for this code is built by tracing the possible ,010
states of the storage devices for all possible inputs. Since 101 I
,100
there are r storage devices and each device can contain at 0 01
most Q different elements, there will be at most 4’ states I I IO
, I I I
in the trellis at any depth. For a reasonable code in which
all encoder states are utilized, the number of trellis states Fig. 6. Trellis at depth h and (k t 1) for k = 4,5, . -. ,lO.
at depth j in the expurgated trellis is given by the formu-
la 0 -0
000 0 -0 _ 0
I I I
number of states in trellis at depth j 000 I
0 0
0010 I I
001 I
0100
q j, j = 1,2,. . . ,r - 1 0101 0 b
01 IO
= 4’9 j = r,r + 1, . . . ,n - r 01 I I
I300
L 4 n-j 2 j = n - r + 1, *. . ,n. 100
1010
I 0

101 I
The trellis is repetitive for j = r + 1, . . . ,n - r. I IO0
I IO I
The general procedure for interconnecting nodes of the I I10
trellis is most easily described by associating with each I I I I ~

node at depth k a polynomial s(x;k) in x of degree (r - 1) Fig. 7. Trellis for depths k = 11,12, * - * ,15.
WOLF: DECODING OF LINEAR BLOCK CODES 79

0000
000 I
0010
001 I
0100
0101 i =I i= 2
01 IO
011 I
IO00
I 00 I
1010
101 I
II00
I IO1 i-3
I I IO
III1
0

Fig. 8. Path in trellis for codeword 10 10 0 10 1 1 10 1 1

i=4

0
0

I
i=5

Fig. 11. Steps in decoding single-parity check code.

Fig. 9. Encoder for single-parity check code.
Portions of the trellis for this code are shown in Figs. 5,
6, and 7. The path corresponding to the codeword 10 10
0 10 1110 110 1 is shown in Fig. 8.
Example 2: The encoder for a binary (n,n - 1) single-
parity check code with generator polynomial g(x) = x +
1 is shown in Fig. 9, and the resultant trellis is shown in Fig.
Fig. 10. Trellis for single-parity check code. 10.
To illustrate the Viterbi decoding algorithm as applied
to this code, we assume that the binary codeword is
with coefficients from GF(q). These polynomials play the transmitted using bipolar signaling over a Gaussian
same roles as the vectors s(k) used previously. The poly- white-noise channel. The input to the decoder is a vector
nomials at depth (k + 1) are then formed from the poly- of n real numbers c = ciczcsc4. * * c,. Using word corre-
nomials at depth k in accordance with the formula lation, the maximum likelihood decoder would form the
sl(x;k + 1) e (xsi(x;k) + xraj) modulog(x). 2n-1 numbers
Rather than pursue this description in abstract detail, we 2 = jcl [Cjxij - Cj(l - Xij)],
i = 1,2, . . . ,2n-1
give two examples.
Example 1: Consider the (15,ll) binary cyclic code with where Xij E (O,l] is the jth component of the ith codeword.
generator polynomial g(x) = 3c4+ x + 1. The correspond- The steps in the Viterbi decoding algorithm for the case
ing encoder is given in Fig. 4. of n = 5 and c = (3,2,-4,-1,4) are shown in Fig. 11. The
For the first 11 clock pulses,the Si and S’sare in position node values V(k) are the circled values at each node. The
one, and the input consistsof the 11 messagedigits. For the maximum likelihood decodedcode word is (1 10 1 1). Note
next four clock pulses, the switches Si and Ss are in posi- that a hard decision decoderwould try to decodethe vector
tion two, and zeros are fed in at the input. The first 11 (1 1 0 0 1) which is equidistant from five different code-
digits which appear at the output are then the 11 message words, and thus would fail to decode.
digits, and the last four digits which appear at the output It should be observed that, in the second step of the
are the four check digits. decoding algorithm, a final decision has been made on the
The state sequence for the encoder for the input se- first digit of the codeword.That is, at this early point in the
quence10100101110is decoding, we have already decided that the first compo-
0000+1100+0110~1111-1011+1001 nent of the codeword is a 1. This is somewhat surpris-
+0100-0010-1101~0110~1111 ing-we have not yet received the parity digit but have
already made a final decision on one of the binary digits!
-1001~0101-0010-0001~0000. The Wagner decoding algorithm [2] which inverts the least
In polynomial notation, each of these states would be likely digit in the hard decision sequenceif the parity check
represented by a polynomial. For example, state 10 0 1 at fails, also makes such final decisions. Indeed, both algo-
depth 11 would be represented by the polynomial s(x;ll) rithms yield maximum likelihood decoding and so must
= 1+ X3. decode to the same codeword. These early final decisions
80 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. m24, NO. 1, JANUARY 1978

are characteristic of Viterbi decoding of arbitrary codes 3) Test if 1 = n2. If so go to step 5);otherwise go to step
and are not limited to just the single-parity check code. 4).
4) Compute, for each (i&, - - - ,i,,)

V. DECODING OF PRODUCT CODES W(i,, i2, . - - , i,,)

In all cases discussed heretofore, the number of states = max [V(l-l)(il + hIl j, i2
needed at some depth in the trellis was equal to qn-“, i
where (n - k) was the number of parity symbols in the + h22j, s. . ,irz + h,,d + &j(l)].
code. For some codes, the maximum number of states can
be much less than qnek; such is the situation for product For each state, retain the sequenceof j values that resulted
codes. in the maximum V. Return to step 2).
Consider a product code with symbols from GF(q) where 5) Compute
the row code is an ( nI,k i) linear code and the column code
is an (n2,k2) linear code. The number of parity symbols is
w-)(0,0, - - * 0) = max [Wz-l)(O,O, * - - 0, j) + Qjtn)].
r = nln2 - k1k2 = k1(n2 - k2) + k2(nl - kl) + (nl- kl)(nz i
- kz). In what follows, we give a decoding algorithm for a
product code that requires only qkl(nz-kz) states. By The sequence of j values that led to the maximum V(“)
symmetry, an algorithm exists with q kz(nl-kl) states. If one (O,O,- - - 0) is the decoded codeword.
code is a low-rate code and the other code is a high-rate 6) Stop.
code, the savings in decoder complexity is enormous when A similar algorithm holds for concatenated codes,where
comparing this algorithm to algorithms which require q’ again the number of required states is much reduced over
or q klkz states. what might be expected from considering the total number
For example, consider a binary product code with a of parity digits transmitted.
(15,5) three-error correcting row code and a (15,14) sin-
gle-error detecting column code. The resultant code is a REFERENCES
(225,70) code with minimum distance 14. The algorithm
111A. J. Viterbi, “Error bounds for convolutional codes and an
described in the previous section would seem to require a asymptoticahy optimum decoding algorithm,” IEEE Trans. Inform.
trellis with 2(22”7e)= 2ls5 states. A decoding algorithm for Theory, vol. IT-13, pp. 260-269, Apr. 1967.
such a trellis is outside the realm of possibility. In the al- PI R. A. Silverman and M. Balser, “Coding for a constant data rate
source,” IRE Trans. Inform. Theory, vol. PGIT-4, pp. 50-63, Jan.
gorithm to follow, only 25 = 32 states would be required in 1954.
the trellis. [31 L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, “Optimal decoding
of linear codes for minimizing symbol error rate,” IEEE Trans.
Decoding Algorithm (Binary Case): Let Q(l) denote the Inform. Theory, vol. IT-20, pp. 284-287, Mar. 1974.
correlation of the 1th row of the received matrix with the 141 C. R. P. Hartmann and L. E. Rudolnh, _ “On ontimum svmbol-by-
codeword from the row code having the information symbol decoding rule for linear codes,” IEEE Trans. Inform.
Theory, vol. IT-22, pp. 514-517, Sept. 1976.
symbols i = (ji,js, * * * ,jk,); j, E (0,l). Let the hij denote
[51 H. Miyakawa and T. Kaneko, “Decoding algorithm for error-cor-
the element in the ith row and the jth column of the parity recting codes by use of analog weights,“Electron. Commun. Japan,
check matrix of the column code. It is assumedthis matrix vol. 58-A, pp. 18-27,1975.
is in echelon canonical form with a unit matrix on the right: Kl D. Chase, “A class of algorithms for decoding block codes with
channel measurement information,” IEEE Trans. Inform. Theory,
hij E (0,l). Let V(l) (ii, is, * . . , i,,) denote the node value vol. IT-18, pp. 170-182, Jan. 1972.
of state (il,i2, - - - ,i,,) after 1 rows of the received matrix [71 E. J. Weldon, Jr., “Decoding block codes on Q-ary output channels,”
IEEE Trans. Inform. Theory, vol. IT-17, pp. 713-718, Nov. 1971.
have been cross-correlated with the codewords in the
PI S. Wainberg and J. K. Wolf, “Algebraic decoding of block codes over
product code. Here i, (a = 1,2, * * . ,r2) is a binary vector a q-ary input, Q-ary output channel, Q > g,” Informat. Contr., vol.
of dimension kl, so there are (2pl)‘z = 2 klrz states. 22, pp. 23-247, Apr. 1973.
[9] B. Dorsch, “Maximum likelihood decoding of binary group codes
Algorithm: 1) Set 1 = 0, for the Gaussian channel,” presented at IEEE Int. Symp. on Inform.
Theory, Noordwijk, The Netherlands, 1970.
W(i,,
0,
i2, - - - , i,,) = _ oD
il = i2.. . = i,, = 0 [lo] G. D. Forney, Jr., “Generalized minimum distance decoding,” IEEE
Trans. Inform. Theory, vol. IT-12, pp. 125-131, Apr: 1966.
> otherwise. [ll] P. Pieper, J. Proakis, R. Reed, and J. K. Wolf, “Maximum likelihood
decoding of concatenated constant weight block codes over a Ray-
2) z--1+1. leigh fading channel,” in preparation.