What is channel coding?

• Channel coding:
– Transforming signals to improve communications
performance by increasing the robustness against
channel impairments (noise, interference, fading, ..)
– Waveform coding: Transforming waveforms to
better waveforms
– Structured sequences: Transforming data
sequences into better sequences, having structured
redundancy.
• “Better” in the sense of making the decision process less
subject to errors.

Slide 2
 Error control techniques
• Automatic Repeat reQuest (ARQ)
– Full-duplex connection, error detection codes
– The receiver sends a feedback to the transmitter, saying that
if any error is detected in the received packet or not (Not-
Acknowledgement (NACK) and Acknowledgement (ACK),
respectively).
– The transmitter retransmits the previously sent packet if it
receives NACK.
• Forward Error Correction (FEC)
– Simplex connection, error correction codes
– The receiver tries to correct some errors
• Hybrid ARQ (ARQ+FEC)
– Full-duplex, error detection and correction codes

Slide 3
 Why using error correction coding?
– Error performance vs. bandwidth
– Power vs. bandwidth
– Data rate vs. bandwidth PB

– Capacity vs. bandwidth Coded

A
Coding gain: F

For a given bit-error probability, C B

the reduction in the Eb/No that can be
realized through the use of code: D
E
 Eb   Eb  Uncoded
G [dB]    [dB]    [dB]
 N0 u  N 0 c Eb / N 0 (dB)

Slide 4
 Channel models
• Discrete memory-less channels
– Discrete input, discrete output
• Binary Symmetric channels
– Binary input, binary output
• Gaussian channels
– Discrete input, continuous output

Slide 5
 Some definitions

• Binary field :
– The set {0,1}, under modulo 2 binary addition and
multiplication forms a field.

Addition Multiplication
00  0 00  0
0 1  1 0 1  0
1 0  1 1 0  0
11  0 1 1  1

– Binary field is also called Galois field, GF(2).

Slide 6
 Some definitions…
• Fields :
– Let F be a set of objects on which two operations ‘+’
and ‘.’ are defined.
– F is said to be a field if and only if
1. F forms a commutative group under + operation. The
additive identity element is labeled “0”.
a, b  F  a  b  b  a  F
2. F-{0} forms a commutative group under . Operation. The
multiplicative identity element is labeled “1”.

3. The operations “+” and “.” distribute:

a, b  F  a  b  b  a  F
a  (b  c)  ( a  b)  (a  c)
Slide 7
 Some definitions…

• Vector space:
– Let V be a set of vectors and F a fields of elements
called scalars. V forms a vector space over F if:
Commutative:
u, v  V  u  v  v  u  F
Distributive:
a  F , v  V  a  v  u  V
Associative:
(a  b)  v  a  v  b  v and a  (u  v )  a  u  a  v
a, b  F , v  V  (a  b)  v  a  (b  v )
v  V, 1 v  v

Slide 8
 Linear block codes

• Linear block code (n,k)

– A set C  Vn with cardinality 2 k is called a linear
block code if, and only if, it is a subspace of the
vector space Vn .
Vk  C  Vn
• Members of C are called code-words.
• The all-zero codeword is a codeword.
• Any linear combination of code-words is a codeword.

Slide 9
 Linear block codes – cont’d

mapping Vn
Vk
C

Bases of C

Slide 10
 Linear block codes – cont’d

• The information bit stream is chopped into blocks of k bits.

• Each block is encoded to a larger block of n bits.
• The coded bits are modulated and sent over channel.
• The reverse procedure is done at the receiver.

Channel
Data block Codeword
encoder
k bits n bits

n-k Redundant bits

k
Rc  Code rate
n

Slide 11
 Linear block codes – cont’d

• The Hamming weight of vector U, denoted by w(U),

is the number of non-zero elements in U.

• The Hamming distance between two vectors U and

V, is the number of elements in which they differ.

• The minimum distance of a block code is

d (U, V )  w(U  V )

d min  min d ( U i , U j )  min w(U i )

i j i

Slide 12
 Linear block codes – cont’d

• Error detection capability is given by

e  d min  1

• Error correcting-capability t of a code, which is defined

as the maximum number of guaranteed correctable
errors per codeword, is

 d min  1
t 
 2 

Slide 13
 Linear block codes – cont’d

• For memory less channels, the probability that

the decoder commits an erroneous decoding is
n
n j
PM     p (1  p) n  j
j t 1  j 
– p is the transition probability or bit error probability over
channel.
• The decoded bit error probability is

1 n
n j n j
PB 
n
 j 
 
j t 1  j 
p (1  p )

Slide 14
 Linear block codes – cont’d
• Discrete, memoryless, symmetric channel model
1 1-p 1
p
Tx. bits Rx. bits
p
0 1-p 0

– Note that for coded systems, the coded bits are modulated and
transmitted over channel. For example, for M-PSK modulation
on AWGN channels (M>2):
2  2log 2 M Ec     2  2log 2 M Eb Rc    
p Q sin     Q sin   
log 2 M  N0  M   log 2 M  N0  M 

where Ec is energy per coded bit, given by Ec  Rc Eb

Slide 15
 Linear block codes –cont’d

mapping Vn
Vk
C

Bases of C
– A matrix G is constructed by taking as its rows the
vectors on the basis, {V1 , V2 , , Vk }
 v 11 v 12  v1 n 
 V1  
   v 21 v 22  v 2 n 
G     
    
 V k   
vk1 vk 2  v kn 

Slide 16
 Linear block codes – cont’d

• Encoding in (n,k) block code

U  mG  V1 
V 
(u1 , u2 , , u n )  (m1 , m2 , , mk )   2 
 
 
Vk 
(u1 , u2 , , u n )  m1  V1  m2  V2    m2  Vk

– The rows of G, are linearly independent.

Slide 17
 Linear block codes – cont’d
• Example: Block code (6,3)
Message vector Codeword

000 000000
 V1   1 1 0 1 0 0  100 110100
G   V 2    0 1 1 0 1 0  010 011010
 V 3   1 0 1 0 0 1  110 1 01 1 1 0
001 1 01 0 0 1
101 0 11 1 0 1
011 1 1 0 011
111 0 0 0 1 11

Slide 18
 Linear block codes – cont’d

• Systematic block code (n,k)

– For a systematic code, the first (or last) k elements in
the codeword are information bits.

G  [P I k ]
I k  k  k identity matrix
Pk  k  (n  k ) matrix

U  (u1 , u 2 ,..., un )  ( p1 , p2 ,..., pn k , m1 , m2 ,..., mk )

      
parity bits message bits

Slide 19
 Linear block codes – cont’d

• For any linear code we can find an matrix H ( n  k )n

, which its rows are orthogonal to rows of G :
T
GH  0
• H is called the parity check matrix and its rows
are linearly independent.
• For systematic linear block codes:
H  [I n  k PT ]

Slide 20
 Linear block codes – cont’d
Data source Format
m Channel U Modulation
encoding
channel
Channel Demodulation
Data sink Format
decoding r Detection
m̂

r  Ue
r  (r1 , r2 ,...., rn ) received codeword or vector
e  (e1 , e2 ,...., en ) error pattern or vector

• Syndrome testing:
– S is syndrome of r, corresponding to the error pattern e.

S  rHT  eHT

Slide 21
 Linear block codes – cont’d

Error pattern Syndrome

000000 000 U  (101110) transmitted.
000001 101
r  (001110) is received.
000010 011
The syndrome of r is computed :
000100 110
001000 001 S  rHT  (001110)H T  (100)
010000 010 Error pattern corresponding to this syndrome is
100000 100
eˆ  (100000)
010001 111
The corrected vector is estimated
Uˆ  r  eˆ  (001110)  (100000)  (101110)

Slide 22
 Hamming codes
• Hamming codes
– Hamming codes are a subclass of linear block codes and belong
to the category of perfect codes.
– Hamming codes are expressed as a function of a single integer
.
m2
Code length : n  2m  1
Number of information bits : k  2 m  m  1
Number of parity bits : n-k  m
Error correction capability : t  1

– The columns of the parity-check matrix, H, consist of all non-zero

binary m-tuples.

Slide 23
 Hamming codes

• Example: Systematic Hamming code (7,4)

1 0 0 0 1 1 1
 
H  0 1 0 1 0 1 1  [I 33 T
P ]
0 0 1 1 1 0 1
0 1 1 1 0 0 0 
1 0 1 0 1 0 0
G   [P I 44 ]
1 1 0 0 0 1 0
 
1 1 1 0 0 0 1 
Slide 24
 Cyclic block codes
• Cyclic codes are a subclass of linear block
codes.
• Encoding and syndrome calculation are
easily performed using feedback shift-
registers.
– Hence, relatively long block codes can be
implemented with a reasonable complexity.
• BCH and Reed-Solomon codes are cyclic
codes.

Slide 25
 Cyclic block codes

• A linear (n,k) code is called a Cyclic code if all

cyclic shifts of a codeword are also a
codeword.
“i” cyclic shifts of U
U  (u0 , u1 , u2 ,..., un 1 )
U (i )  (un i , un i 1 ,..., un 1 , u0 , u1 , u2 ,..., un i 1 )
– Example:

U  (1101)
U (1)  (1110) U ( 2)  (0111) U (3)  (1011) U ( 4)  (1101)  U

Slide 26
 Cyclic block codes

• Syndrome decoding for Cyclic codes:

– Received codeword in polynomial form is given by

Received r ( X )  U ( X )  e( X ) Error
codeword pattern

– The syndrome is the reminder obtained by dividing the received

polynomial by the generator polynomial.

r ( X )  q ( X )g ( X )  S ( X ) Syndrome
– With syndrome and Standard array, error is estimated.

• In Cyclic codes, the size of standard array is considerably reduced.

Slide 27
 Example of the block codes

PB
8PSK

QPSK

Eb / N 0 [dB]
Slide 28