MODERN DIGITAL COMMUNICATION

Discrete Stationary Sources

- Dr. P. Susheelkumar S,
- Faculty – Dept. of Electronics Engineering
- Datta Meghe College of Engineering
- Airoli, Navi Mumbai
Discrete Stationary Sources:
LZ 78 Approach:

1) LZ 77 approach assumes that like patterns will occur close together.

2) It makes use of this past of the sequence as the dictionary for
encoding.
3) But, for any pattern that recurs over a period of time longer than
that covered by the coder window, the pattern will not be captured
4) The worst case scenario would be where the sequence to be
encoded was periodic with a period longer than than the search
Eg: (a Sequence with a period of 9 & a Search Buffer of length 8)

abcdefghia b c d e f g h i a b c d e f g hi
Search Buffer Lookahead buffer

1) In the case above , none of the new symbols will have a match in
the search buffer and will have to be represented by separate
codewords.
2) If the search buffer would have been 1-symbol longer then we
would have achieved significant compression
3) In this case, we see a net expansion instead of compression.
Discrete Stationary Sources:
LZ 78 Approach:

1) LZ 78 addresses this problem by dropping the reliance on the

search buffer and keeping an explicit dictionary.
2) This dictionary is built at both the encoder and the decoder and
care should be taken to build the dictionary in an identical manner
at both ends,
3) In this the inputs are coded as a double (i, c) , with i being an index
number corresponding to the dictionary entry that was the
longest match to the input and c being the code for the character
in the input following the matched portion of the input
4) As in LZ77, if there is no match then index value of 0 is used. This
double becomes the newest entry in the dictionary.
 Discrete Stationary Sources:
1) Encode the sequence using LZ 78 approach.

1) Initially the Dictionary is empty

2) Furst few symbols encountered
are encoded with index – 0
3) 1st three encoder o/ps are:
(0,C(w)) , (0,C(a)) , (0,C(b))
4) 4th symbol is b which is at entry
3 or index-3, also if we append
the next symbol, the new
pattern is not present in the
dictionary.
5) Hence we encode these two
symbols i.e. the pattern ba as
(3,C(a)) and adds the patter ba
as the 4th entry in the
dictionary.
 Discrete Stationary Sources:
1) Encode the sequence using LZ 78 approach.

1) Continuing in this fashion , the

encoder output and the
dictionary is developed as
shown besides.
2) Thus entries in the dictionary
keep getting longer
3) Drawback: The dictionary keeps
growing longer and longer.
4) So practically we have to stop
the growth of dictionary at
some stage or prune it back or
treat the encoding as a fixed
dictionary scheme.
Discrete Stationary Sources:
LZW Algorithm:

1) This technique removes the necessity of encoding the 2nd

element of the pair (i, c) as in LZ78 approach.
2) The encoder will only send the index to the dictionary.
3) So the dictionary has to be primed with all the letters of the
source alphabet.
4) The input to the encoder is accumulated in a pattern p as long
as p is contained in the dictionary.
5) If the addition of another letter a results in a pattern - pa -
that is not in the dictionary, then the pattern -pa- is added
into the dictionary and the index of -p- is transmitted to the
receiver.
6) And we start with the letter -a- and check out for patterns
existing in the dictionary as previously done.
Discrete Stationary Sources:
Encode the sequence using LZW approach.

1) Since the Alphabet Letters are known beforehand itself , let the
initial LZW dictionary look like:
Discrete Stationary Sources:
Encode the sequence using LZW approach.

1) Since the Alphabet Letters are known beforehand itself , the

final LZW dictionary look like:
Discrete Stationary Sources:
Decoding the Sequence received:

1) !st received code is : 5

so the decoded symbol is: w after refering the Pre-laid table

1) 2nd Recved Code is: 2

So the decoded symbol is: a and the dictionary table is
appended by : wa ( concatenating earlier pattern ‘w’ by ‘a’) as
its 6th element and: a is the current pattern.

2) 3rd received code is: 3

So the decoded symbol is : b and the dictionary table is
appended by : ab as its 7th element and now: b becomes the
current pattern
3) 4th receivedcode is : 3
So the decoded symbol is: b and the dictionary table is
appended by: bb as its 8th element and: b is the current
pattern now.
Discrete Stationary Sources:
Decoding the Sequence received:

5) 5th received symbol is: 2

So the decoded symbol is: a and the dictionary table is
appended by : ba as its 9th element
6) 6th Received code is: 1
So the decoded symbol is : b and the dictionary table is
appended by: ab as its 10th element
7) 7th received code is: 6
So the decoded symbol is: wa and in this case we
concatenate the earlier symbol i.e. ‘b’ with the 1st symbol of
the pair: wa i.e. we get: bw which is not there in the table.
Hence we append the table by symbol : bw as its 11th element
(And symbol w is taken as the existing pattern for the next
decode) Now the current pattern: w is appended by the 2nd
symbol of the pair: wa , but since it is present in the table we
proceed to the next received code with the existing pattern :
Discrete Stationary Sources:
Decoding the Sequence received:

8) 8th received code is: 8

So the decoded symbol is: bb and the earlier pattern: wa
is appended by the 1st symbol of the pair to get: wab which is
not there in the dictionary table, hence the table is append by
the symbol triplet: wab as its 12th element. Hence the
current pattern is: b which is appended by the second symbol
of the decoded symbol i.e. : b to get: bb which is already
present in the table. Therefore the current pattern becomes:
bb for the next decode.
9) 9th received code is: 10
So the decoded symbol is: ab and we check for: bba in the
dictionary , but since its not there the table is appended by:
bba as its 13th element. Now ‘a’ remains the existing pattern
which is concatenated by b to get: ab which is already present
in the Table. Hence the current pattern becomes: ab for the
Discrete Stationary Sources:
Decoding the Sequence received:

10) 10th received code is: 12

So the decoded symbol is: wab , we check for : abw
which is not there in table. Hence the table is appended by
abw which is the 14th element. Now: w is the current pattern
which is concateated by the second symbol of the triplet to get:
wa which is present in the table. Hence wa becomes the
current pattern which is appended by the 3rd symbol in the
decoded symbols to get: wab , which is present in the table.
Hence wab is taken as the current pattern and we move to the
next decode.
11) 11th received code is: 9
So the decoded symbol is: ba and the current pattern
wab gets concatenatedby the 1st symbol to get: wabb which
is added as the 15th element. The current pattern becomes: b
which is appended by :a to get ba which is already in the table.
Discrete Stationary Sources:
Decoding the Sequence received:

12) 12th received code is: 11

So the decoded symbol is: bw and accordingly we check
for: ba b in the table, since it is not there, we append the table
by adding : ba b as its 16th element. Now b is the current
pattern which is concatenated by: w to give: bw which is
present in the table. Hence the current pattern becomes: bw
and we go for the next decode.
13) 13th received code is: 7
So the decoded symbol is: ab and accodringly we check
for: bwa in the table. Since it is not present, bwa becomes
the 17th element of the table. And: a is the current pattern
which is concatenated with b to give: ab which is also present
in the table. Hence: ab forms the current pattern and we go
for the next decode.
Discrete Stationary Sources:
Decoding the Sequence received:

14) 14th received code is: 16

So the decoded symbol is: ba b and accordingly we check
for: abb in the table but since it is not there, abb becomes the
18th element and: b becomes the current pattern which is
concatenated with: a to give: ba which is present in the
table. Hence: ba becomes the current pattern available, which
is concatenated with : b to give: ba b which is present in the
table. Hence ba b becomes the current pattern and we go for
the next decode.
15) 15th received code is: 5
So the decoded symbol is: w and accordingly we
concatenate ba b with: w to give: ba bw which is not present
in the table. Hence ba bw is added as the 19th element of the
table and: w becomes the current pattern and we go for the
next decode.
Discrete Stationary Sources:
Decoding the Sequence received:

16) 16th received code is: 4

So the decoded symbol is: o and accordingly w is
concatenated with: o to give: wo which becomes the 20th
element of the table and: o becomes the current pattern.
Further we go for the next decode.
17) 17th received code is: 4
So the decoded symbol is: o and accordingly o is
concatenated with: o to get: oo which is added as the 21st
element of the table and: o is the current pattern and we go for
the next decode.
18) 18th received code is: 11
So the decoded symbol is: bw and accordingly o b is added
as the 22nd element of the table and further we concatenate b
with: w to get bw which is already present.
Discrete Stationary Sources:
Decoding the Sequence received:

19) 19th received code is: 21

So the decoded symbol is: oo and accordingly bwo is
added as the 23rd element and the current patter becomes: o
which is concatenated with: o to get: oo which is already
present in the table and hence the current pattern.
20) 20th received code is: 23
So the decoded symbol is: bwo and accordingly oo b is
added as the 24th element of the table and again b is
concatenated with: w to get bw which is already present in
the table. Hence bw is further concatenated with: o to get:
bwo which is already present in the table. Hence the current
pattern becomes: bwo and we go for the next decode.
21) 21st received symbol is: 4
So the decoded symbol is: o and accordingly we
concatenate bwo with: o to get: bwoo which is added as the
th
Discrete Stationary Sources:
Decoding the Sequence received:

19) 19th received code is: 21

So the decoded symbol is: oo and accordingly bwo is
added as the 23rd element and the current patter becomes: o
which is concatenated with: o to get: oo which is already
present in the table and hence the current pattern.
20) 20th received code is: 23
So the decoded symbol is: bwo and accordingly oo b is
added as the 24th element of the table and again b is
concatenated with: w to get bw which is already present in
the table. Hence bw is further concatenated with: o to get:
bwo which is already present in the table. Hence the current
pattern becomes: bwo and we go for the next decode.
21) 21st received symbol is: 4
So the decoded symbol is: o and accordingly we
concatenate bwo with: o to get: bwoo which is added as the
th