FORWARD ERROR CORRECTION [FEC]

ERROR CORRECTING CODES

BLOCK CODES LINEAR CODES

CONVOLUTIONAL CODES NON LINEAR CODES
 AUTOMATIC REPEAT REQUEST SYSTEM [ARQ]

From To
Source destination
 LINEAR BLOCK CODES

(n , k) LINEAR BLOCK CODE

k MESSAGE BITS 2k DIFFERENT MESSAGES

n–k PARITY BITS

n CODEWORD BITS 2n DIFFERENT COMBINATIONS

BUT ONLY 2k ARE VALID CODEWORDS AND

THE REMAINING 2n - 2k ARE INVALID CODEWORDS.

SOURCE INFORMATION RATE , Rs

CHANNEL DATA RATE , R0 = (n/k) Rs

CODE RATE , rc = k/n

SYSTEMATIC CODES
 LINEAR BLOCK CODES

(n,k) Linear Block Code

(m0,m1,………………..,mk-1) is a block of k message bits.
2k distinct message blocks. k=4 24=16 message blocks

When a message is applied to Linear block encoder,

it produces n-bit codeword (c0,c1,
…………………………….,cn-1).

2n distinct codewords n=7 27=128 distinct codewords

but only 2k codewords (16) are valid codewords
and 2n-2k codewords (112) are invalid codewords

Then (b0,b1,…………bn-k-1) denote (n-k) parity bits. 5

 Systematic structure of codeword
c0,c1,………….cn-k-1 , cn-k,cn-k+1,….………….,cn-1
C = b0,b1,…………….…bn-k-1 m0 , m1,………..……………..,mk-1
Parity bits Message bits

where ci = -- (1)

and the (n-k) parity bits are linear sums of k message bits as

-- (2)
bi = pi,0m0 + pi,1m1+ ……………………..+ pi,k-1mk-1.
for i = 0,1,…………….,n-k-1.
where coefficients pi,j are defined as
-- (3)
; where j = 0,1,…..,k-1 6
 Matrix Notation

Message vector m = [ m0,m1,………………..,mk-1 ] 1 x k --(4)

Parity vector b = [ b0,b1,…………bn-k-1 ] 1 x (n – k) --(5)

Code vector C = [ c0,c1,…………………..………….,cn-1 ] 1 x n --(6)

where b = 1xmk) * kx(n-k)

1x(n-k) P --
(7)

and

Coefficient Matrix P = --
k x (n-k)
(8)

7
from (4), (5) & (6) we can write C as a partitioned vector as

C =[ b m ] --(9)
1x n
1x (n-k) 1x k
Substituting (7) in (9), we can write

C =[ m P m ]
1x n
1x k kx(n- k) 1x k

C =[ m [ P Ik ] ] 1x n --(10)
1x k kx(n- k) kx k

where Ik is (k x k) identity matrix

Ik = --(11)
8
Defining the ( k x n ) Generator matrix G as

G =[ P Ik ] --(12)
kxn
kx(n- k) kx k

Then we can write eqn (10) as

Code word C=[ m * G ] --(13)

1x n
1x k kx n

Closure Property : Sum of any two codewords is another

codeword.
Ci + Cj = miG +mjG = (mi+ mj)G = mkG = Ck
where Ck is another codeword.

The linearity property of a linear block code means that the

linear combination of any two codewords is another codeword.
9
 Another way of expressing relationship between
message bits and parity bits

Defining the (n-k) x n Parity check matrix H as

H = [ In-k PT ] (n-k) x n
--(14)
(n-k) x (n-k) (n-k) x k

where
PT is transpose of coefficient matrix P and
In-k is (n-k) x(n-k) identity matrix
Then (n-k) x k
[ H * GT ] (n-k) x k = [ In-k PT ] (n-k) x n kxk
(n-k) x n nxk (n-k) x (n-k) (n-k) x k
nxk
= PT + PT = O
where ‘O’ is an (n-k) x k matrix with all zero elements
Hence H GT = O or equivalently G HT = O. --(15)
10
Multiplying both sides of equation (13) by HT and using
equation (15), we get

C HT = m G HT --(16)

C HT = O [matrix of size (1 x n-k)]

The matrix H is called Parity Check Matrix and the equations
specified by equation (16) are called Parity Check equations.

11
 Syndrome Decoding

Received vector W = C + e , where e is error vector

and e = [ e0, e1,…………., en-1 ] 1x n

where,

The receiver task is to decode code vector C from the

received vector W.
The decoding operation starts with computation of a 1 x (n-k)
vector called Error syndrone vector or Syndrome
Syndrome is defined as
S = W HT --(17)
S = [ s0, s1, ……, sn-k-1 ] 1x (n-k) 12
 Properties of Syndrome
Property 1 : The Syndrome depends only on the error pattern
and not on the transmitted codeword.
S = WHT = ( C + e ) HT

S = CHT + eHT
S = eHT since CHT = O
Property 2 : All error patterns that differ at most by a
codeword have the same syndrome.
for k message bits – there are 2k (n-bit)codewords denoted
Ci, i = 0,1,…..,(2k-1).
We have Wi = e + Ci ; i = 0,1,…..,(2k-1) -- Coset of the code
The (n,k) LBC has 2(n-k) possible cosets. Then

Wi HT = eHT + CiHT = eHT since CiHT = O &

13
W H = eH = S
T T
Property 3 : The syndrome S is the sum of those columns of
the matrix H corresponding to the error locations.
We have H = [ h1, h2,…………., hn ]

S = e HT = [ e1, e2,………, en ]

S=

Where is ith row of the matrix HT and

ei is the ith element of error pattern e, where

substituting ei , we get S equal to those rows of the matrix HT

corresponding to the error locations in e.
Rows in matrix HT are equal to columns of the matrix H 14
Property 4 : with syndrome decoding, an (n,k) LBC can
correct upto t errors per codeword, provided that n and k
satisfy the Hamming bound

where is a binomial coefficient , namely

A binary code for which Hamming bound is satisfied with

equality sign is called Perfect Code
15
 Standard Array
The arrangement of the n-tuples with 2k columns and 2n-k rows is
called as Standard Array for the given (n,k) linear code.
The rows are called Cosets.
The left most n-tuple of each row is called Coset Leader.

16
A (6,3) linear block code can correct all single errors.
Assuming a generator matrix

[G] =

The Standard Array for the (6,3) LBC is given below.

17
 Standard Array for (6,3) LBC

Syn- Coset n-tuples

drome Leader
Code
000 000000 110001 101010 011011 011100 101101 110110 000111
vectors
110 000001 110000 101011 011010 011101 101100 110111 000110
101 000010 110011 101000 011110 011001 101111 110100 000101
011 000100 110101 101110 011000 011111 101001 110010 000011
001 001000 111001 100010 010100 010011 100101 111110 001111
Cosets
010 010000 100001 111010 001100 001011 111101 100110 010111
100 100000 010001 001010 111100 111011 001101 010110 100111

111 001001 111000 100011 010101 010010 100100 111111 001110

18
 Table-Lookup Decoding Algorithm

1. For received vector W, Compute S = W HT.

2. Take the coset leader for the syndrome S from decoding
table.
3. For getting correct codeword, add the coset leader to the
received vector.

19
20
 Minimum distance considerations
Hamming Distance : d( X , Y ) number of bit positions X and Y differ

Hamming Weight : w (X) number of non-zero elements in X

Minimum Distance : dmin smallest hamming distance between any

two codewords of the code and is equal to the minimum Hamming weight of
the non-zero codewords in the code.

Error detecting capability of a block code with minimum

distance dmin is ( dmin – 1)

An (n,k) LBC can correct all error patterns of weight t or less if

and only if
d (Ci , Cj) 2 t + 1 for all Ci and Cj 21
An (n,k) LBC of minimum distance dmin can correct upto t errors,
if and only if

where denote largest integer no greater than the enclosed

number.
The code is referred as a t – error correcting code.

22
r (7, 4) Hamming code, n = 7, k = 4, n – k = 3 and with G given as below

nimum distance dmin = 3

ror detecting capability = 2

ror correcting capability t = 1
amming codes are single error correcting binary perfect codes 23
 m
m0 m1 m2 mk-1

m0 m1 mk-1 m0 m1 mk-1 m0 m1 mk-1

c0 c1 cn-k-1

24
ci =

bi = pi,0m0 + pi,1m1+ ……………………..+ pi,k-1mk-1.

for i = 0,1,…………….,n-k-1.

; where j = 0,1,…..,k-1

25
Input W
w0 w1 w2 wn-2 wn-1

w0 wn-k wn-k+1 wn-1 w1 wn-k wn-k+1 wn-1 wn-k-1 wn-k wn-k+1 wn-1

26
H = [ In-k PT ]

HT =

s = WHT = [w0, w1,…wn-k-1,wn-k,wn-k+1,….wn-1]

Hence
wn-k = m0, wn-k+1 = m1, ………, wn-1 = mk-1

&
si = wi + pi,0m0 + pi,1m1+ ……………………..+ pi,k-1mk-1.
for i = 0,1,…………….,(n-k-1).
27
W

w0 w1 wn-1

w0 w1 wn-1

c0 c1 cn-1

28
Problem :The Generator matrix for (7 , 4) LBC is given as

(i) Write all the possible code-words and minimum distance.

(ii) Draw the Encoder.
(iii) Let the received codeword is 1 0 0 1 1 1 1. Verify
whether it is a valid codeword or not ? If not, what is the
correct codeword.
(iv) Draw the syndrome circuit diagram.
(v)Draw the error pattern detecting circuit
(vi) Draw the decoder 29
Solution : G =[ P Ik ]

(i) k = 4 , n = 7 and (n-k) = 3

C=mG
[ c 0 c1 c2 c3 c4 c5 c6 ] = [ m 0 m 1 m 2 m 3 ]

Encoding expressions :
c0 = m0+m2+m3 c1 = m0+m1+m2 c2 = m1+m2+m3
c3 = m0 c4 = m1 c5 = m2 c6 = m3

Substituting the message bits into the encoding expressions,

we can obtain the codeword bits.

Hence the various possible 7- bit codewords for various 4-bit

message blocks are given in the table below . 30
 Message bits Codeword bits
m 0 m1 m 2 m3 c0 c 1 c 2 c3 c 4 c 5 c 6 Hamming weight

0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 1 1 0 1 0 0 0 1 3
0 0 1 0 1 1 1 0 0 1 0 4
0 0 1 1 0 1 0 0 0 1 1 3
0 1 0 0 0 1 1 0 1 0 0 3
0 1 0 1 1 1 0 0 1 0 1 4
0 1 1 0 1 0 0 0 1 1 0 3
0 1 1 1 0 0 1 0 1 1 1 4
1 0 0 0 1 1 0 1 0 0 0 3
1 0 0 1 0 1 1 1 0 0 1 4
1 0 1 0 0 0 1 1 0 1 0 4
1 0 1 1 1 0 0 1 0 1 1 4
1 1 0 0 1 0 1 1 1 0 0 4
1 1 0 1 0 0 0 1 1 0 1 3
1 1 1 0 0 1 0 1 1 1 0 4
1 1 1 1 1 1 1 1 1 1 1 7

nimum distance is defined as the smallest hamming weight of the

So, the Minimum distance = 3
n zero code vectors in the code.
31
 (ii) Encoder Diagram :

m
Input
Message
Message Register

m0 m1 m2 m3

To Channel
C
Output
Code word
m0 m1 m2
m3 m2 m0 m2 m1 m3

b0 b1 b2

c0 c1 c2
Parity Register
32
(iii) The Received codevector is W = 1 0 0 1 1 1 1

To verify whether W is valid or Invalid codevector and if

invalid to get the correct codevector, we need to find H, HT,
Syndrome expressions, Syndrome for W = 1 0 0 1 1 1 1,
Decoding table and the correct codevector.

We have G = [ P Ik ] and H = [ In-k PT ]

Using G matrix we can write

33
Hence

We have S = W HT = [ w0 w1 w2 w3 w4 w5 w6 ]

and Syndrome expressions are

s0 = w0 + w3 + w5 + w6

s1 = w1 + w3 + w4 + w5

s2 = w2 + w4 + w5 + w6
34
Given W = [ w0 w1 w2 w3 w4 w5 w6 ] = [ 1 0 0 1 1 1 1 ]

Using the syndrome equations

The Syndrome S = WHT = [ 0 1 1 ] which is non zero and

also it is the 5th row in HT (i.e., error is in 5th bit of W).

Hence the received vector W is invalid codevector.

Decoding Table : Syndrome Coset Leader/ Error pattern

s0 s1 s2 e0 e1 e2 e3 e4 e5 e6
0 0 0 0 0 0 0 0 0 0
1 0 0 1 0 0 0 0 0 0
0 1 0 0 1 0 0 0 0 0
0 0 1 0 0 1 0 0 0 0
1 1 0 0 0 0 1 0 0 0
0 1 1 0 0 0 0 1 0 0
1 1 1 0 0 0 0 0 1 0
1 0 1 0 0 0 0 0 0 1 35
From decoding table the error pattern for S = [ 0 1 1 ]

is e = [ 0 0 0 0 1 0 0 ]

The correct codeword is C = W + e

C= [1001111]+[0000100]

C= [1001011]

36
 (iv) Syndrome circuit :

Input W w4 w5 w6
w0 w1 w2 w3

w3 w5 w3 w4 w4 w5
w0 w6 w1 w5 w2 w6

s0 s1 s2
37
(v) From decoding table, we can write the expressions for
error bits as Syndrome Coset Leader/ Error pattern
s0 s1 s2 e0 e1 e2 e3 e4 e5 e6
e0 = s0 s1 s2 0 0 0 0 0 0 0 0 0 0
e1 = s0 s1 s2
1 0 0 1 0 0 0 0 0 0
e2 = s0 s1 s2 0 1 0 0 1 0 0 0 0 0

e3 = s0 s1 s2 0 0 1 0 0 1 0 0 0 0
1 1 0 0 0 0 1 0 0 0
e4 = s0 s1 s2
0 1 1 0 0 0 0 1 0 0
e5 = s0 s1 s2 1 1 1 0 0 0 0 0 1 0

e6 = s0 s1 s2 1 0 1 0 0 0 0 0 0 1

38
 Error pattern circuit :

s0 s1 s2

e0 e1 e2 e3 e4 e5 e6

w0 w1 w2 w3 w4 w5 w6

c0 c1 c2 c3 c4 c5 c6
Corrected Codeword 39
 (vi) Decoder :
W w0 w1 w2 w3 w4 w5 w6
Received
Codeword

Syndrome s0 s1 s2

Error
Pattern e0 e1 e2 e3 e4 e5 e6

w0 w1 w2 w3 w4 w5 w6
c0 c1 c2 c3 c4 c5 c6
Corrected Codeword 40
41
C=[m b]
Since G = [Ik p]

42
43
44
For a systematic linear block code (6,3) three parity check digits are given by :
C4 =d1 ⊕d2⊕d3
C5 =d1 ⊕d2
C6 =d1 ⊕d3
Construct generator matrix, prepare a suitable decoding table.

Given a (7,4) linear block code whose generator matrix is given by

G=

Find the Parity check matrix and code word that begins with 1011?

45
END OF LBC

46