Signal Compression

Dr. Waquar Ahmad

National Institute of Technology Calicut

[email protected]

November 15, 2021

Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 1 / 17

Transform Coding-Introduction

Transform coding uses transforms to obtain representations of the signal that aid
compression.
In particular, we look at transforms that represent signals in the frequency domain.
Humans perceive signals in only a narrow band of frequencies—spatial frequencies, in the
case of images, and temporal frequencies, in the case of audio.
We use these limitations to discard or deemphasize components of the signal that may
not be perceived, or not be as important for perception.
Transforms allow us to find representations of signals with correlation structures which
compact most of the signal energy into a few components.
This then allows us to not expend resources on the components with little signal energy -
thus obtaining compression.

Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 2 / 17

example
Height Weight
65 170
75 180
60 150
70 170
56 130
80 203
68 160
50 110
40 80
50 153
69 148
62 140
76 164
64 120
Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 3 / 17
Example

look at the height and weight as the coordinates of a point in two-dimensional space.

Notice that the output values tend to cluster around the line y = 2.5x.
We can rotate this set of values by the transformation

θ = Ax

Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 4 / 17

Example

where x is the two-dimensional source output vector

 
x
x= 0
x1

x0 corresponds to height and x1 corresponds to weight, A is the rotation matrix.

 
cos φ sin φ
A=
− sin φ cos φ

φ is the angle between the x-axis and the y = 2.5x line, and
 
θ
= 0
θ1

is the rotated or transformed set of values.

Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 5 / 17
Example
The matrix A is  
0.37139068 0.92847669
A=
0.92847669 0.37139068
The transformed sequence (rounded to the nearest integer) is shown in the table

Dr. Waquar Ahmad (NITC) Figure: Caption

Lecture-6 November 15, 2021 6 / 17
Transform

The transforms that we will deal will be linear transforms. that is, we can get the
sequence {θn } from the sequence {xn } as
N−1
X
θn = xi an,i .
i=0

This is referred to as the forward transform.

A measure of the differing characteristics of the different elements of the transformed
sequence {θn } is the variance σn 2 of each element. These variances will strongly influence
how we encode the transformed sequence.

Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 7 / 17

Transform

The original sequence {xn } can be recovered from the transformed sequence {θn } via the
inverse transform :
N−1
X
xn = θi bn,i
i=0

The transforms can be written in matrix form as

θ = Ax

x = Bθ

Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 8 / 17

KARHUNEN–LOÉVE TRANSFORM

The best-known datadependent transform is the discrete Karhunen–Loéve transform

(KLT).
The rows of the discrete Karhunen–Loéve transform, also known as the Hotelling
transform, consist of the eigenvectors of the autocorrelation matrix.
The autocorrelation matrix for a random process X is a matrix whose (i, j)th element
[R]i,j is given by
[R]i,j = E [Xn , Xn+|i−j| ]
A transform constructed in this manner will minimize the geometric mean of the variance
of the transform coefficients.
Hence, the Karhunen–Loéve transform provides the largest transform coding gain of any
transform coding method.
Do example 13.4.1 from the Khalid Sayyood book

Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 9 / 17

DISCRETE COSINE TRANSFORM

The discrete cosine transform (DCT) gets its name from the fact that the rows of the
NxN transform matrix C are obtained as a function of cosines.
 q
 1
cos (2j+1)iπ i = 0, j = 0, 1, 2, ....N − 1
[ci,j ] = qN 2N
 2
cos (2j+1)iπ i = 0, 1, 2, ....N − 1, j = 0, 1, 2, ....N − 1
N 2N

The DCT is closely related to the discrete Fourier transform (DFT) and, in fact, can be
obtained from the DFT. However, in terms of compression, the DCT performs better than
the DFT.

Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 10 / 17

DISCRETE WALSH–HADAMARD TRANSFORM

A transform that is especially simple to implement is the discrete Walsh–Hadamard

transform (DWHT).
A Hadamard matrix of order N is defined as an NxN matrix H, with the property that
HH T = NI , where I is the NxN identity matrix.
Hadamard matrices whose dimensions are a power of two can be constructed in the
following manner:  
HN HN
H2N =
HN H−N
with H1 = [1]. Therefore    
H 1 H1 1 1
H2 = =
H1 H−1 1 −1

Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 11 / 17

DISCRETE WALSH–HADAMARD TRANSFORM
 
  1 1 1 1
H2 H2 1 −1 1 −1
H4 = =
1 1 −1 −1

H2 H−2
1 −1 −1 1

The DWHT transform matrix H can be obtained from the Hadamard matrix by
multiplying it by a normalizing factor so that HH T = I instead of NI and by reordering
the rows in increasing order of sequence.
The sequence of a row is half the number of sign q changes in that row.
Normalization involves multiplying the matrix by N1 .
Because the matrix without the scaling factor consists of 1, the transform operation
consists simply of addition and subtraction.
So this transformation is useful when computation power is less.
However, the amount of energy compaction obtained is substantially less than that of the
DCT. .
Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 12 / 17
QUANTIZATION AND CODING OF TRANSFORM
COEFFICIENTS
If the amount of information conveyed by each coefficient is different, Then assign
differing numbers of bits to the different coefficients.
There are two approaches to assigning bits. , First approach relies on the average
properties of the transform coefficients, while the second approach assigns bits as needed
by individual transform coefficients.
In the first approach, we first obtain an estimate of the variances of the transform
coefficients.
These estimates can be used by one of two algorithms to assign the number of bits used
to quantize each of the coefficients.
We assume that the relative variance of the coefficients corresponds to the amount of
information contained in each coefficient.
Thus, coefficients with higher variance are assigned more bits than coefficients with
smaller variance.
Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 13 / 17
QUANTIZATION AND CODING OF TRANSFORM
COEFFICIENTS
Let us find an expression for the distortion, then find the bit allocation that minimizes the
distortion.
If the average number of bits per sample to be used by the transform coding system is R
and the average number of bits per sample used by the kth coefficient is Rk , then
M
1 X
R= Rk (1)
M
k=1

where M is the number of transform coefficients.

2 is related to the kth quantizer
The reconstruction error variance for the kth quantizer σrk
2
input variance σθk by the following:

σr2k = αk 2−2Rk σθ2k

where αk is a factor that depends on the input distribution and the quantizer.
Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 14 / 17
QUANTIZATION AND CODING OF TRANSFORM
COEFFICIENTS
The total reconstruction error is given by
M
X
σr2 = αk 2−2Rk σθ2k (2)
k=1

The objective of the bit allocation procedure is to find Rk to minimize Equation 2 subject
to the constraint of Equation 1.
If we assume that αk is a constant α for all k, then the minimization problem in terms of
Lagrange multipliers is
M M
!
X 1 X
J=α 2−2Rk σθ2k − λ R − Rk (3)
M
k=1 k=1

Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 15 / 17

QUANTIZATION AND CODING OF TRANSFORM
COEFFICIENTS
Taking the derivative of J with respect to Rk and setting it equal to zero, we can obtain
this expression for Rk
 
1 2 1 λ
Rk = log2 (2α ln 2σθk ) − log2 (4)
2 2 M
Substituting this expression for Rk in Equation 1, we get a value for λ/M:
λ 1
2 M −2R
= ΠM
k=1 (2α ln 2σk ) 2
M
Substituting this expression for λ/M in Equation 4, we finally obtain the individual bit
allocations
1 σθ2k
Rk = R + log2 1
2 ΠM 2 M
k=1 (σθk )
Although these values of Rk will minimize reconstruction error, they are not guaranteed
to be integers or even positive.
Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 16 / 17
QUANTIZATION AND CODING OF TRANSFORM
COEFFICIENTS
The standard approach at this point is to set the negative Rk s to zero. This will increase
the average bit rate above the constraint.
Therefore, the nonzero Rk s are uniformly reduced until the average rate is equal to R.
The following bit allocation algorithm can be obtained.
1 Compute σr k for each coefficient. As the number of bits allocated to each coefficient is zero,
initially,σr k = σθ k .
2 Set Rk = 0 for all k; and set Rb = MR, where Rb is the total number of bits available for
distribution.
3 Sort the standard deviations σr k . Suppose σr l is the maximum.
4 Increment Rl by 1, and divide σr l by 2.
5 Decrement Rb by 1. If Rb = 0, then stop; otherwise, go to step 3.
If we follow this procedure, we end up allocating more bits to the coefficients with higher
variance. This form of bit allocation is called zonal sampling.

Dr. Waquar Ahmad (NITC) Lecture-6 November 15, 2021 17 / 17