Scribd
Upload
12 views5 pages

Information Theory and Coding Variable Length Code

Uploaded by

himanshupatlan191
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
12 views5 pages

Information Theory and Coding Variable Length Code

Uploaded by

himanshupatlan191
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Information Theory and Coding

Variable length code

‫ فؤاد حمادي‬.‫م‬
9191 - 9102
2- Variable length code

The codes we have looked above have all fixed lengths and they are
called block code from the fact that the message, are of fixed block
lengths in the stream of symbols being sent.

We now examine variable length code in more detail. The advantage


of a code where the message symbols are of variable length is that some
time the code is more efficient in the sense that to represent the same
information we can use fewer digits on the average. To accomplish this
we need to know something about the statistics of the message being sent.
If every symbol is as likely as every other one, then the fixed length code
are about as efficient as any code can be. But if some symbols are more
probable than others, then we can take advantage of this feature to make
the most frequent symbols correspond to the shorter encodings and the
rare symbols correspond to the longer encodings. This is exactly the
idea behind variable length coding.

However, variable length code bring with them a fundamental


problem, at the receiving end, how do you recognize each symbol of the
code? In, for example, a binary system how do you recognize the end of
one code word and the beginning of the next ?

If the probabilities of the frequencies of occurrence of the


individual symbols are sufficiently different , then variable – length
encoding can be significantly more efficient than fixed – length encoding
Note: for each symbol increase the probability decrease the code length

1. Source coding for special source

Source coding can achieve 100 % efficiency when r* -level code is


used with source having a probability in the form P(x i)= r -Ii ( where Ii is
an integer ) for all xi .

Ex1: design a binary code for the following source :


Ex2: A source produce 7 symbols x1,x2,… ,x7, with probabilities 0.0625,
0.25, 0.125, 0.25, 0.125, 0.0625, 0.125. design a binary code for the
above source, then determine the code efficiency.

Shannon – Fano method

To encode a message using Shannon-Fano method, you can follow


the below steps :

1. Sort the symbols in descending order according to their probabilities.

2. Divide the list of symbols into two parts : upper and lower, so that the
summation of the probabilities of the upper part is equal as possible to the
summation of the lower part symbols.
3. Assign "0" code to each of the upper part symbols, and "1" code to
each of the lower part symbols.

4. Divide each of the upper and lower part into upper and lower
subdivision as in step (2) above, and assign the code "0" and "1" as in
step (3) above.

5. Continue in step(4) until each subdivision contains only one symbols.

Ex1: A source produce 5 independent symbols ( x1, x2, x3, x4, x5 ) with its
corresponding probabilities 0.1, 0.3, 0.15, 0.25, 0.2 . design a binary code
for the above source symbol using Shannon – fanon method.

You might also like