Scribd
Upload
31 views

Cdmtcs Research Series: Ludwig Staiger

This document presents a proof that for any admissible infinite length distribution, there exists a maximal prefix code whose codewords satisfy that length distribution. It begins by introducing prefix codes and stating Kraft's inequality, which establishes the existence of prefix codes for certain finite length distributions. It then proves a generalization of Kraft's inequality, showing that a simple modification of Kraft's construction can be used to construct infinite maximal prefix codes whenever the length distribution sums to 1. An algorithm called MaxKraft is presented that constructs such a code by iteratively adding codewords.

Uploaded by

Fais Abdul
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
31 views

Cdmtcs Research Series: Ludwig Staiger

This document presents a proof that for any admissible infinite length distribution, there exists a maximal prefix code whose codewords satisfy that length distribution. It begins by introducing prefix codes and stating Kraft's inequality, which establishes the existence of prefix codes for certain finite length distributions. It then proves a generalization of Kraft's inequality, showing that a simple modification of Kraft's construction can be used to construct infinite maximal prefix codes whenever the length distribution sums to 1. An algorithm called MaxKraft is presented that constructs such a code by iteratively adding codewords.

Uploaded by

Fais Abdul
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

CDMTCS

Research
Report
Series

On Maximal Prefix Codes

Ludwig Staiger
Martin-Luther-Universität
Halle-Wittenberg

CDMTCS-280
May 2006

Centre for Discrete Mathematics and


Theoretical Computer Science
O M P C
Ludwig Staiger
Martin-Luther-Universität Halle-Wittenberg
Institut für Informatik
von-Seckendorff-Platz 1
D–06099 Halle (Saale), Germany
[email protected]
Abstract
Kraft’s inequality is a classical theorem in Information Theory which
establishes the existence of prefix codes for certain (admissible) length dis-
tributions. We prove the following generalisation of Kraft’s theorem: For
every admissible infinite length distribution one can construct a maximal
prefix codes whose codewords satisfy this length distribution.

Prefix codes are widely used in data transmission or in (algorithmic) information


theory (see [3, 4]). A set of nonempty words C ⊆ X ∗ over an alphabet X is called
a prefix code provided w ∈ C is not a prefix of v ∈ C, for every pair of distinct
words w, v ∈ C.
A classical theorem about the existence prefix codes is called Kraft’s inequal-
ity [2].
Theorem 1 (Kraft’s inequality). Let X be a finite alphabet, I ⊆ N and let f :
I → N be a non-decreasing function such that n∈I |X|− f (n) ≤ 1. Then there is a
P
prefix code C = {vn : n ∈ I} ⊆ X ∗ such that |vn | = f (n).
Here |X| denotes the cardinality of the set X, and |v| denotes the length of the
word v and n∈I |X|− f (n) ≤ 1 means that the length distribution f : I → N is
P
admissible.
The aim of this note is to show that a simple modification of Kraft’s construc-
tion (see e.g. [4]) is suitable for the construction of infinite maximal prefix codes
C ⊆ X ∗ whenever v∈C |X|−|v| ≤ 1.
P
Here a code C ⊆ X ∗ is referred to as maximal prefix if C is a prefix code and
for every prefix code C 0 ⊇ C implies C 0 = C. It is known that a maximal prefix
code need not be maximal as a code (see e.g. [1, II. Example 3.1]). For finite
codes C ⊆ X ∗ , however, a maximal prefix code satisfies v∈C |X|−|v| = 1 and is also
P
maximal as a code.
|X|− f (n) ≤
P
Theorem 2. Let f : N → N be a non-decreasing function such that
n∈N
1. Then there is a maximal prefix code C = {vn : n ∈ N} ⊆ X ∗ such that |vn | = f (n).
We use the following characterisation of maximal prefix codes whose proof is
given here for the sake of completeness.
Lemma 3. Let M be an infinite subset of N. A code C ⊆ X ∗ is maximal prefix if
and only if for all w ∈ {v : v ∈ X ∗ ∧ |v| ∈ M} there is a v ∈ C such that w v v or
v v w.
Proof. If C is not maximal prefix then there is a w < C such that C ∪ {w} is a
prefix code. Consider wu ∈ X ∗ where |wu| ∈ M. Since w @ v and v @ w for every
v ∈ C, the same holds true for the word wu.
Conversely, if for some w ∈ {v : v ∈ X ∗ ∧ |v| ∈ M} there is no v ∈ C such that
w v v or v v w then C ∪ {w} is a prefix code properly containing C. 
Now, using this lemma we construct a prefix code which satisfies the condition
of Lemma 3 for some infinite set M ⊆ { f (n) : n ∈ N}. This is done by the
following algorithm MaxKraft.

Algorithm MaxKraft
0 n := 0 ; l := 0 ; C := ∅ ; M := ∅
1 For i = 1 to ∞ do
2 l := f (n) ; W := X l \ C · X ∗ ; M := M ∪ {l}
3 Let W = {w1 , . . . , w|W| }
4 For j = 0 to |W| − 1 do
5 C := C ∪ {w j+1 · 0 f (n+ j)−l }
6 Endfor
7 n := n + |W|
8 Endfor
Here the set M is included just to have a reference to Lemma 3.
At stage i + 1 our parameters before constructing the new approximation Ci+1
are Ci , ni and li+1 = f (ni ) where f (ni − 1) = sup{|w| : w ∈ Ci }.
Then the set Wi+1 = X li+1 \ Ci · X ∗ is the set of words which have no prefix in
Ci . For each of the words {w1 , . . . , w|Wi+1 | }, the body of the For-loop (lines 4 to 6)
adds the word w j+1 · 0 f (ni+1 + j)−li+1 of length f (ni+1 + j) to Ci . Thus f ( j) is the length
of the jth word in Ci+1 if j ≤ |Ci+1 |, in particular f (ni+1 − 1) = sup{|w| : w ∈ Ci+1 }.
As in the proof of Kraft’s inequality, we obtain that
X |Ci |
X
|Wi+1 | = |X|li+1 −|v|
= |X|li+1
· |X|− f ( j) < |X|li+1 .
v∈Ci j=1
Consequently, the algorithm does not stop, that is, Ci ⊂ Ci+1 , and returns an
infinite set C = ∞
S
i=1 C i in which the word constructed in step j has length f ( j).
Clearly, the resulting Ci+1 is a prefix-code, if Ci is a prefix-code, and by the
steps in lines 4 and 5 every word of length li+1 has a prefix in Ci ⊆ Ci+1 or is a
prefix of some word in Ci+1 .
At the next stage this process is repeated for the new (greater) length li+2 :=
f (ni+1 + |Wi+1 |). So, by induction, it is seen that C = ∞
S
i=1 C i is a prefix code for
which the infinite set M = {li : i = 1, . . . } is a witness for its prefix maximality.
The algorithm depends on the monotonicity of the function f : N → N. The
monotonicity guarantees that, when, at some stage i, the finite approximation Ci
of the code C is constructed, all words w ∈ C \ Ci will have length |w| ≥ f (ni − 1).

References
[1] J. Berstel and D. Perrin. Theory of Codes. Academic Press, 1985.
[2] L.G. Kraft. A Device for Quantizing Grouping and Coding Amplitude Modulated
Pulses, MS Thesis, Electrical Eng. Dept., MIT, Cambridge, Ma., 1949.
[3] M. Li and P.M.B. Vitányi, An Introduction to Kolmogorov Complexity and its Appli-
cations. Springer-Verlag, New York, 1993.
[4] R. Johannesson, Informationstheorie, Addison-Wesley, 1992.

You might also like