Lecture 1

Introduction to Error Control Coding

2/03/2010

Introduction

Outline
What is error control coding? Why is error control coding important? How does error control coding work? Where does error control coding fit in? Encoding Decoding Performance Analysis BSC Channel Bandwidth efficiency vs power efficiency.
2/03/2010 Introduction 2

What is Error Control Coding?

Most communication channels or storage mediums suffer from noise, interference or distortion

The goal of error control coding is to encode information in such a way what even if the channel or storage medium introduces errors, the receiver can correct the errors and recover the original transmission.

2/03/2010

Introduction

Why is error control coding important? (1/3)

(1) Reliable Data Communication Helpful in any scenario where two nodes are communicating via a transmission medium Example 1: Deep Space Communications Reed Solomon Codes Example 2: Digital Satellite TV Reed Solomon Codes
2/03/2010 Introduction 4

Why is error control coding important? (2/3)

(2) Reliable Data Storage Helpful for any digital storage medium, such as CDS, DVDS, Example 1: CDS, DVDs, Blu-Ray Discs Reed Solomon Codes Example 2: Modern Hard Drives Reed Solomon Codes
can correct 2.4 mm in length

2/03/2010

Introduction

Why is error control coding important? (3/3)

(3) Reduce Transmit Power Useful for any transmitting devices requiring low power Example 1: Space Probes Reed Muller Codes, Reed Solomon Codes

2/03/2010

Introduction

How does error control coding work? (1/4)

Add properly designed redundancy to each data message. These redundant digits are used for detecting and/or correcting transmission or storage errors Example: Original Data Message = 0 1 0 1 Coded Message =01011101 How do we use these bits to control the error?

2/03/2010

Introduction

How does error control coding work? (2/4)

How do we use the redundancy bits to control the error? (1) Automatic Repeat Request (ARQ) Redundant bits used to detect errors If error occurs, then the receiver requests retransmission from the transmitter Request signal sent through feedback channel Transmission occurs until no more errors are detected

2/03/2010

Introduction

How does error control coding work? (3/4)

How do we use the redundancy bits to control the error? (2) Error Correcting Code Redundant bits used to correct errors

2/03/2010

Introduction

How does error control coding work? (4/4)

Which one should we use ARQ or Error Correcting Codes? ARQ Advantage: Simple Disadvantage: Feedback channel is needed Error Correcting Code Advantage: No feedback channel Disadvantage: Complex

2/03/2010

Introduction

10

Outline
What is error control coding? Why is error control coding important? How does error control coding work? Where does error control coding fit in? Encoding Decoding Performance Analysis BSC Channel Bandwidth efficiency vs power efficiency.
2/03/2010 Introduction 11

Where does Error Correcting Codes fit in? (1/3)

Information Source Source Encoder Channel Encoder Modulator

Digital Channel

(Writing Unit)

Digital Data Source

Noise

Transmission Channel (Storage Medium)

Digital Data Destination

Destination

Source Decoder

Channel Decoder

Demodulator
(Reading Unit)

Fig. 3.3 Block diagram of a typical data transmission or storage system

2/03/2010 Introduction 12

Where does Error Correcting Codes fit in? (2/3)

Example 1: Wireless file transfer application from Bob to Alice (1) Digital Data Source Bobs file on laptop, composed of binary bits (2) Digital Channel Modulation Wireless Channel (transmission medium) Demodulation (3) Digital Data Destination Bobs received file on Alices laptop Problem wireless channel produces errors Solution error control coding
2/03/2010 Introduction 13

Where does Error Correcting Codes fit in? (3/3)

Example 2: Bob wants to give mp3s to Alice (1) Digital Data Source MP3 (2) Digital Channel CD (3) Digital Data Destination Alice receives CD from Bob Problem: scratches on CD Solution Error Control Coding

2/03/2010

Introduction

14

Encoding (1/5)
Aim: Add redundant bits to original data message to form coded message k digits are encoded into a sequence of n digits, called a codeword. In general n>k k message length n code length R = k/n code rate Classified into two main types Block Codes codes of fixed length without memory Convolutional Codes codes with memory

2/03/2010

Introduction

15

Encoding (2/5)
Block codes a message of k digits is mapped into a structured sequence of n digits Properties N-k redundant bits are added to each message for protection against errors Each encoding operation is independent (no memory) of past encodings. Example: k=3, n=6 Data Message = 100 111 Coded Message = 0 1 1 1 0 0 0 0 0 1 1 1 In general, both message and coded message are binary symbols, 0 and 1. There are thus 2^k distinct messages and 2^k corresponding binary codewords

2/03/2010

Introduction

16

Encoding (3/5)
Example 1.1: Let k = 3 and n = 6. The following table gives a block code of length 6. The code rate is R = 1/2.
Message Codeword

(0 0 0) (1 0 0) (0 1 0) (1 1 0) (0 0 1) (1 0 1) (0 1 1) (1 1 1)

( 0 0 0 0 0 0) (0 1 1 1 0 0) (1 0 1 0 1 0) (1 1 0 1 1 0) (1 1 0 0 0 1) (1 0 1 1 0 1) (0 1 1 0 1 1) (0 0 0 1 1 1)

2/03/2010

Introduction

17

Encoding (4/5)
Convolutional codes each k digits is mapped into a n digit coded block Properties The n-digit coded block depends not only on the input k-digit message block but also on m ( 1) previous message blocks. That is, the encoder has memory of order m. The collection of all possible code sequences is called an (n,k,m) convolutional code

2/03/2010

Introduction

18

Encoding (5/5)
Example 1.2: Let k = 1, n = 2 and m = 2. The following circuit generates a (2,1,2) convolutional code.
+
vl(1)

cl

cl 1

cl 2

Shift register stage

+
+
vl( 2)

exclusive-OR gate

Input: (Data Message) Outputs: (Coded Message)

ct
vl(1) = cl + cl 2 vl( 2) = cl + cl 1 + cl 2 .

2/03/2010

Introduction

19

Decoding (1/3)
Aim: Recover the original coded message from the received sequence of bits In more detail: Suppose a codeword corresponding to a certain message is transmitted over a noisy channel. Let r be the corresponding received sequence. The receiver (or decoder), based on r, the encoding rules and the noise characteristics of the channel, makes a decision which message was actually transmitted. This decision making operation is called decoding. The device which performs the decoding operation is called a decoder.
2/03/2010 Introduction 20

Decoding (2/3)
c
v Channel Encoder v Digital Channel

Channel Decoder

Suppose the codeword v is transmitted. Let r be the corresponding output of the digital channel. The decoder must produce an estimate of the message v based on r. Decoding error occurs if v v 1-1 relationship between c and v, means c c A decoding rule is a strategy for choosing an estimated message for each possible received sequence . Obviously, we would like to devise a decoding rule such that the probability of a decoding error is minimised. Such a decoding rule is called an optimum decoding rule.
2/03/2010 Introduction 21

Decoding (3/3)
Suppose all the messages are equally likely. An optimum decoding can be done as follows. For every codeword compute the conditional probability P( r | v j ). The codeword with the largest conditional probability is chosen as the estimate for the transmitted codeword. This decoding rule is also called the maximum likelihood decoding (MLD). MLD Advantage: lowest possible decoding error MLD Disadvantage: May be highly complex Goal: find low complex decoding schemes to achieve as close as MLD as possible Some low complex schemes capable of achieving MLD performance in certain scenarios.
2/03/2010 Introduction 22

Outline
What is error control coding? Why is error control coding important? How does error control coding work? Where does error control coding fit in? Encoding Decoding Performance Analysis BSC Channel Bandwidth efficiency vs power efficiency.
2/03/2010 Introduction 23

Performance Analysis (1/2)

Once we have devised encoding/decoding schemes, we want to analyse the performance. This can be used to answer important questions, such as (1) How much better is our coded scheme over an uncoded scheme? (2) How do we compare the performance of different encoding/decoding schemes? We consider the bit error rate (BER) For uncoded transmission with BPSK modulation
2/03/2010 Introduction 24

Performance Analysis (2/2)

(1) How much better is our coded scheme over an uncoded scheme? The BER allows us to obtain the coding gain, which is a measure of the difference between signal-to-noise ratios (SNR) between the uncoded system and the coded system to reach the same level of BER.

2/03/2010

Introduction

25

BSC Channel (1/2)

c
v Channel Encoder v Digital Channel

Channel Decoder

How do we model the digital channel? One common way is Binary Symmetric Channel (BSC) If the channel is an additive white Gaussian noise (AWGN) channel, hard decision made by the demodulator results in a BSC.
1-p 0 0

p=probability of error
p

1 1-p

2/03/2010

Introduction

26

BSC Channel (2/2)

Advantages Simple to analyse performance Many problems in communication theory can be reduced to the BSC Can lead to useful insights into more complicated channels
1-p 0 0

1 1-p

2/03/2010

Introduction

27

Bandwidth Efficiency vs Power Efficiency (1/2)

In most of coded digital communication systems, coding is performed separately from modulation - demodulation. Error control is provided by transmitting additional redundant bits in the code, which has the effect of lowering the information bit rate per channel bandwidth. In this case, bandwidth efficiency is traded for increased power efficiency. This is suitable for power-limited systems. However, when bandwidth efficiency is a major concern (such as in bandwidth-limited systems), the most effective method for error control is to combine coding and modulation as a single entity.
2/03/2010 Introduction 28

Bandwidth Efficiency vs Power Efficiency (2/2)

In such an approach, coding is redefined as a process of imposing certain patterns on the transmitted signal. The combined modulation coding is called coded modulation. This error control technique is most suitable for band-width - limited systems where the available bandwidth must be utilised effectively.

2/03/2010

Introduction

29

Summary of Lecture (1/2) What is error control coding? Why is error control coding important? How does error control coding work? Where does error control coding fit in?

2/03/2010

Introduction

30

Summary of Lecture (2/2)

Encoding Add redundant bits to form coded message from original data message Block codes and convolutional codes Decoding Coded message goes through channel which adds errors to form the received message Decoding attempts to recover the original data message from the received message Goal is to find low-complexity schemes to achieve close to MLD

Performance Analysis BSC Channel Bandwidth efficiency vs power efficiency.

2/03/2010

Introduction

31

Future lectures
Focus on different encoding/decoding schemes Each encoding/decoding scheme has different properties which make it suitable for different needs of applications Application Needs: low complexity, good performance We will try to understand these properties, encoding and decoding proccesses. Need understanding of Binary Field and Vector Spaces (Lecture 2)
2/03/2010 Introduction 32