DIGITAL SIGNAL PROCESSING

Prof. Dr. Ljubia Stankovi

University of Montenegro, Montenegro
Digital signal processing
Digital signal processing
CHAPTERS:
1. Discrete-time signals and systems
2. Discrete Fourier transform
3. z-transform
4. Spectral estimation
5. Time-frequency analysis
6. Multidimensional signal processing
Discrete-Time signals and systems
Discrete signals
Discrete signal x[n] can be represented as a
sequence of real or complex numbers, where with
n is denoted n th number.
Some important and very often used discrete
signals (sequence):
Unite impulse
Unite impulse is called also delta function or
simply impulse, and defined as:
Discrete-Time signals and systems
Unite step
A discrete time unite step function is defined to be:
Real exponential functions
A real discrete-time exponential function is given
by:
Discrete-Time signals and systems
Sinusoidal and complex exponential signals
Periodicity
A signal x[n] is said to be periodic with N if x[n] =
x[n+N]. A complex exponential sequence is
periodic only if 2/
0
is integer number or if it is
rational number p/q.
0
jw n
e
Discrete-Time signals and systems
Examples
Solutions:
a)
for k = 1 N = 3.
Discrete-Time signals and systems
b)
for k =7 we have N = 6.
c)
From above equation we can conclude that
sequence is not periodic.
Discrete-Time signals and systems
It is important to note that an arbitrary sequence
can be written in the form:
Some important definition for a discrete signals
1. Magnitude of signal:
2. Energy of a signal:
Discrete-Time signals and systems
Linear shift-invariant discrete systems
Discrete systems can be described by transform
which maps output sequence x[n] into output
equence y[n]:
where T is operator which denote transform of a
system.
Discrete-Time signals and systems
Linear system
System is linear if:
Examples:
Consider following systems and check their linearity:
Discrete-Time signals and systems
Solutions:
Thus, the system is not linear.
Thus, the system is linear.
Discrete-Time signals and systems
Shift invariance
A system is shift invariant if the characteristics of
system are not function of time, i.e. if a input signal
x(n) produce output signal is y(n), than input
signal x(n- N) will produce output signal y(n N)
Examples
Consider systems:
Discrete-Time signals and systems
Solutions:
Obviously, the system is shift-invariant.
The system is not shift-invariant.
Discrete-Time signals and systems
Description of a linear shift-invariant systems through
a response of a systems on the unite impulse:
If a systems are shift-invariant than:
This is convolution sum and it can be written as:
It is very easy to show:
Discrete-Time signals and systems
Causality and stability
System is causal if :
System is stabile if for we have
for any value n.
Proof:
From above equation follows:
Discrete-Time signals and systems
Difference equations
The general form of linear difference equations
with constant coefficients is presented by:
For this equation there is exist family of solutions,
so it is necessary to set initial conditions for
uniquely solution. If we assume that system is
causal i.e. if x(n) = 0 for n < N than y(n) = 0 for n <
N, we can write:
Discrete-Time signals and systems
Example:
Consider difference equation in the form:
We will find impulse response if we take
and assume that system is causal
Discrete-Time signals and systems
This is an example of a system with infinite impulse
response-IIR system.
If we take Aj = 0 for j > 0 than in our example we
have:
This system has finite impulse response-FIR system.
Discrete-Time signals and systems
Fourier transform of a discrete signals
Consider complex exponential signal:
For this signal we can write:
With relation:
is defined Fourier transform of a discrete signal h(n).
Discrete-Time signals and systems
Thus:
In general case:
Uniform convergence:
Having in mind:
We can conclude that Fourier transform of discrete
signals is periodic with 2.
Since is periodic we can consider it as a
Fouriers series.
Discrete-Time signals and systems
A inverse transform can be obtained:
We have:
Examples:
By using definition compute Fourier transform for:
Discrete-Time signals and systems
Solutions:
1.
2.
In this case the amplitude characteristic will be:
Discrete-Time signals and systems
Properties of the Fourier transform of discrete
signals
1. Linearity
2. Shifting in the time
3. Modulation
Discrete-Time signals and systems
4. Convolution
By substitution m = n k we obtain:
Discrete-Time signals and systems
5. Product
If we interchange order of integration and
summation, we have:
Finally we have:
Discrete-Time signals and systems
Sampling of continuous-time signals
According to Shannon theorem continuous signal
can be recovered from its discrete version if
discrete signal has been sampled by periodic
sampling T which satisfy condition:
T =1/(2fm)
where fm is maximal frequency of a signal.
Consider an analog signal x
a
(t) that has the
Fourier representation:
Discrete-Time signals and systems
Assume that signal xa(t) has limited bandwidth,
i.e.:
Consider now a periodic form X
p
(j
a
) of the X
a
(j
a
)
that has a period 2
0
. The Fourier transform
X
a
(j
a
) can be recovered from X
p
(j
a
) if
0

a
.
Since X
p
(j
a
) is periodic, it can be expanded in the
Taylor series:
Discrete-Time signals and systems
where is T =/
0
. The Fouriers coefficient can be
obtained from:
If we compare last equation with equations that
give x
a
(t), we can conclude:
Finally, we have that samples of signal x
a
(t) and
X
p
(j
a
) form a Fourier pair.
Now, we have:
Discrete-Time signals and systems
Discrete-Time signals and systems
Examples:
1. If output signal is:
and impulse response of a system is:
Compute signal on the output of the system.
Solution:
On the output of the system amplitude and phase
of the input signal will be changed.
Discrete-Time signals and systems
Namely:
Discrete-Time signals and systems
2. If:
find signal h(n).
Solution:
Discrete-Time signals and systems
3. Find the Fourier transform of the signal
h(t) = e
t
u(t) and draw its amplitude characteristic.
Write a discrete form of the signal and draw its
amplitude characteristics for the cases T = 1 and
T =0. 2.
Solution:
Thus, we have
By discretization we obtain:
and its Fourier transform:
Discrete-Time signals and systems
4. Compute the sum:
Consider the sequence with the Fourier transform:
From above equation follows:
Discrete Fourier Transform
Definition of the discrete Fourier transform
We have seen that the Fourier transform of a
discrete signal is continual and periodic function
with period 2. If we want to use this transform in
digital signal processing, we need its discrete
version, i.e. we have to sample it in frequency
domain.
Consider the Fourier transform of a signal x(n).
Assume that X(e
j
) is sampled with rate = 2/N,
where N is number of samples along the period.
Discrete Fourier Transform
Since samples of signal and its Fourier transform
are the transformations pair, we have that
sampling in the frequency domain cause periodical
signal in the time domain and vice versa.
Thus, for the discrete Fourier transform we have
the periodic signal x
p
(n) obtained from x(n).
For the sampling rate in frequency domain
(
a
= /T)
we have periodic series in the time domain with
the period t
p
= N
p
T:
From the above equation we have:
Np = N
It means the following: If we want that the
periodic signal contains the original signal x(n),
discretization must be done with the same (or
greater) number of samples as a duration of the
signal x(n).
Discrete Fourier Transform
Definition of the discrete Fourier transform:
where it is assumed:
X
p
(k) is the discrete Fourier transform.
Discrete Fourier Transform
If we use notation:
the discrete Fourier transform can be written in the
form:
Or matrix form:
We see that the computation of the Discrete
Fourier Transform require approximately N
2
multiplications and additions.
Discrete Fourier Transform
Inverse discrete Fourier transform
Inverse form of the discrete Fourier transform can
be obtained by multiplication of DFT definition by
W
N
km
. Thus we have:
Taking:
Discrete Fourier Transform
From the above equations follows the definition of
the inverse Fourier transform:
If the duration of the signal x(n) is smaller than
N we have:
Discrete Fourier Transform
i.e.
where w(n) denotes the window function defined
by:
Discrete Fourier Transform
Examples
1. Find the discrete Fourier transform of the
sequence:
Solution:
From the equation for x(n) we see that the
duration of x(n) is N
p
= 5. Thus we have to
use N N
p
=5. Taking N = 5 we have:
The function Xp(k) is periodic with N = 5 as well.
Discrete Fourier Transform
2. If we have the periodic sequence X
p
(n) with
period N
p
= N, and X
p
(k) is its DTF, find the DFT for
the same sequence, taking N
p
= 2N.
Solution
for N
p
=2N we have:
the above equation can be written in the form:
Discrete Fourier Transform
because x(n) = x(n + N) and W
2N
kN
=(1)
k
.
Now we can write relation between X
p
(k) and
X
p
(k):
Xp (k) = 2X
p
(k) for k even, and X
p
(k) = 0 for k odd.
Discrete Fourier Transform
Relationship between frequency and k th
number in the discrete Fourier transform
If the signal x(n) is obtained by sampling of the
analog signal x
a
(t ), than the frequency of the
discrete signal can be represented through the
analog frequency
a
, by:
By discretization of the Fourier transform we have:
From the previous two equations we have:
Discrete Fourier Transform
Note that this equation holds only for k N/2 1.
Frequencies between N/2 1 and N are mapped
negative frequencies:
Discrete Fourier Transform
Zero padding
Number of samples of the discrete Fourier
transform N in frequency domain depends on the
number of samples of a signal in the time domain.
If we want to get more samples within the basic
period of the Fourier transform (interpolation), by
using the discrete Fourier transform, then it is
necessary to take more samples as a signal period.
Discrete Fourier Transform
It can be easy obtained by adding zero values at
end of the signal sequence.
Number of zero values depends from our will, i.e.,
on how many samples we want to have in the
discrete Fourier transform.
This procedure can be understood as an
interpolation of the discrete Fourier transform.
Discrete Fourier Transform
Some properties of the discrete Fourier transform -
Convolution of periodic signals
Consider following properties of the discrete
Fourier transform:
1. If :
Then:
We have the sequence:
Discrete Fourier Transform
By substitution n m = l follows:
It can be shown that:
Discrete Fourier Transform
Convolution of periodic versions of signals
Consider signals x
p1
(n) and x
p2
(n) which are
periodic versions of signals x
1
(n) and x
2
(n).
In order to derive period for this convolution
consider the following example.
Illustrative example
In this example we will consider signals:
x
1
(n) = u(n) u(n 5), and x
2
(n) = u(n) u(n 5).
Discrete Fourier Transform
Since signals have period N = 5, then for the
DFT calculation, we can form periodicals versions
with N 5. In this example assume N = 7.
Discrete Fourier Transform
From the Figure it is easy to obtain
x
p3
(n) = x
p1
(n) x
p2
(n):
From the Figures we see that results are
different for the original signals and their periodic
versions.
The reason is in overlapping of fictive periods.
Discrete Fourier Transform
If we want to avoid this effect we have to
introduce enough number of zero values.
Namely, if either sequence has duration N then
the period has to be 2N-1, or in general if duration
of the first one is N and duration of the second one
is M, then the period has to be M N1.
Discrete Fourier Transform
When the duration of input signal is
significantly different from the duration of
sequence of impulse response (duration of the
impulse response is significantly shorter), we can
decompose the input sequence into few
subsequence, i.e.
Discrete Fourier Transform
Discrete Fourier Transform
Thus output is obtained as:
Here we must be careful since before every
convolution we must add L 1 zero values (it is
assumed that L duration of a subsequence).
Discrete Fourier Transform
Fast Fourier transform FFT
Algorithm called the Fast Fourier transform or FFT
algorithm plays very important role in digital signal
processing.
This algorithm is an interesting research topic.
That is the reason why exist various forms of this
algorithm.
By using the FFT time needed for computation
of the discrete Fourier transform can significantly
be reduced comparing time for computation of the
discrete Fourier transform by definition.
Discrete Fourier Transform
In our consideration we will present an
approach that belongs to the group of algorithms
called Decimations-in-Frequency.
The aim of this decimation is to decompose
X
p
(k) Into subsequences, then further,
subsequences into subsequences etc.
For this algorithm it is necessary that the
number of samples is of the form:
Discrete Fourier Transform
Now decompose sequence X
p
(k) into two
sequences:
Discrete Fourier Transform
From the previous equation we have:
Having in mind:
and that summations in both terms are from 0 to
N/2 1, we can write:
Discrete Fourier Transform
If we separate the previous equation for k = 2r and
k = 2r + 1, we get:
Where:
and
Discrete Fourier Transform
Since:
we have:
We see that the resulting transform is in the
form of two transforms with N/2 terms.
Discrete Fourier Transform
Thus, one discrete Fourier transform with N
terms is decomposed into two discrete Fourier
transform with N/2 terms.
We have concluded that for the calculation of
the DFT with N elements, by definition, we need
approximately N2 operations. For two DFTs of N/2
elements we need 2(N/2)
2
= N
2
/2 calculations.
Discrete Fourier Transform
This procedure can be continued in n steps, we
have the elements as a simple multiplication and
summation.
Consider an example with N = 2
3
.
Illustration
Finally, we can conclude that:
N log
2
N
is number of necessary multiplications and
summations, as well.
Discrete Fourier Transform
It is interesting to find ratio N
2
/(N log
2
N) which
illustrates efficiency of the FFT algorithm in
comparison with calculation of the discrete Fourier
transform by definition.
For example, for N = 512, if we need 1 minute
to compute the discrete Fourier transform on a
computer, by using the FFT it will be calculated for
only 1 second.
Discrete Fourier Transform
Examples
1. Find the discrete Fourier transform of the
following sequences:
a) x(0) = 1, x(1) = 1, x(2) = 1
b) x(n) = a
n
(u(n) u(n N))
Solutions:
a) Taking N = 3 we can write:
Discrete Fourier Transform
b) Taking period N we have:
Taking for example a =1, follows:
Discrete Fourier Transform
2. Find relationship between X
p
(k) and X
p
(N k) in
the case of real sequences x
p
(n).
Solution
Since:
We can conclude that in the case of real
sequence x
p
(n) we have:
Discrete Fourier Transform
3. If g(n) and f(n) are real sequences, show that
their discrete Fourier transforms (G(k) and
F(k)) can be obtained from the discrete Fourier
transform Y(k) of the sequence y(n) = g(n)+ jf(n).
Solution
From the signal y(n) = g(n) + jf(n) we can write:
The discrete Fourier transform of y(n) is:
Discrete Fourier Transform
Conjugate complex value of the previous
equation gives:
From the above equation the discrete Fourier
transform of the signal y

(n) follows:
Discrete Fourier Transform
4. The relationship between k th number in the
discrete Fourier transform and analog value of the
frequency is given by:
If we want to avoid shifting of the discrete Fourier
transform for k = N/2 1 we can multiply input
signal x(n) by (1).
Proof:
The discrete Fourier transform of the (1)x(n)
is:
Discrete Fourier Transform
for k N/2 1 we have:
Thus we have:
For the case k > N/2 1 follows:
Thus we have
Illustration additionally can show results of this
transformation.
Z
Z

Transform
Transform
The z-transform can be understood as a
generalization of the Fourier transform.
Applications of this transform are mainly for
description and realization of systems.
Definition of the z-transform
Z-transform of the signal x(n) is defined as:
where z is complex.
Z
Z

Transform
Transform
X(z) is defined for z where previous sum converges.
The region of convergence of the z-transform is
defined by two annular ring with r
1
and r
2
which
contain the poles of the function X(z).
The values r
1
and r
2
depend from the behavior
of the signal x(n) in the cases when n tends plus
infinity and minus infinity.
Z
Z

Transform
Transform
Example 1
Find the z-transform of the signal x(n)=u(n)
Solution:
According to definition we have:
We know that previous sum converge for
|z
1
| <1 i.e. |z| > 1.
Z
Z

Transform
Transform
Thus the region of convergence is exterior (to the
pole location z = 1) of the unit circle |z| = 1. The
poles are denoted by x , while the zeros by o .
Z
Z

Transform
Transform
Example 2
Find the z-transform of the signal x(n) = u(n 1).
Solution:
According to definition we have:
where the region of convergence is defined by
|z| < 1.
Z
Z

Transform
Transform
From the previous two examples we can conclude
that either have the same X(z).
Thus we can conclude that by using z-transform a
signal is not uniquely determined. However if we
have also the region of convergence uniquely will
be satisfied.
Consider now, four important sequence and find
their z-transform.
1. Causal series x(n) = 0 for n < 0
The z-transform of this signal is:
Z
Z

Transform
Transform
We see that z belong to the region of
convergence.
Thus we can conclude that region of convergence
will be annular ring exterior to the pole location
with the longest distance R from origin, so we
have: R < |z| < .
Z
Z

Transform
Transform
2. Non causal series x(n) = 0 for n > 0.
We se that sum converge for z = 0. The region
of convergence is the disk centered at the origin
and interior to the pole location R. Where R is the
pole the nearest to the origin, 0 |z| < R.
Z
Z

Transform
Transform
3. Sum of the causal and anticausal series
For this case we have:
From the previous considerations we have
concluded that first series converge for:
0 |z| < R1
and second one for:
R2 < |z| <
Z
Z

Transform
Transform
The resultant region of convergence is:
R2 < |z| < R1
This is the annular ring. If R2 > R1 than the region
of convergence is .
Z
Z

Transform
Transform
Example
Find the z-transform and the region of
convergence for the series:
X(n) = a
n
u(n) b
n
u(n 1).
Solution
The first sum converge for |z| >a, while the
second one for |z| < b. thus the region of
convergence is:
a < |z| < b
Z
Z

Transform
Transform
4. Finite length sequences x(n) = 0 for n n1 and
n n2
We conclude that sum converge for any z except 0
and/or what depends from conditions are
n
1
and n
2
positive or negative numbers.
Z
Z

Transform
Transform
Inverse z-transform
The inverse z-transform is defined by:
If we multiply right and left side of the previous
equations by: z
k1
and if we perform integration
along restricted closed path C which resides
within the region of convergence, we obtain:
Z
Z

Transform
Transform
Since integral on the right side of the equation is
different from zero only for k = n, we
have:
This is the general form for determination of the
inverse z-transform. The previous integral can
be calculated by using the theorem of residuum:
The residuum of the function F(z) in the pole
z = z
0
, that is pole of order k, can be calculated
with:
Z
Z

Transform
Transform
The inverse z-transform will be calculated on the
base of expansion of X(z) in the series with
respect z1. In that case we write X(z) in the form:
Than by comparing the previous equation and
definition of the inverse z-transform we see that
X(n) = X
n
.
Z
Z

Transform
Transform
Example 1.
Find the inverse z-transform for:
Solution:
Expanding the previous equations into series for
|z| > 1/4 we have:
Thus we can conclude:
X(n) = (1/4)
n
u(n)
Z
Z

Transform
Transform
In the case when the region of convergence is
|z| < 1/4 coefficients of series must be less
than 1, so X(z) has to be transformed in the form:
Thus,
Z
Z

Transform
Transform
Example 2.
Find x(n) if
Solution
Consider first:
thus we have:
x(n) = a
n
u(n)
Z
Z

Transform
Transform
If we find differential of X
1
(z), we obtain:
Now we have:
x(n) = a
n-1
u(n)
Z
Z

Transform
Transform
Table of the z-transform
Z
Z

Transform
Transform
Properties of the z-transform
Derivations of the properties of the z-transform are
analogy with the properties of the Fourier
transform.
1. Linearity
If we have y(n) = ax(n) + bh(n) than
Y(z) = aX(z) + bY(z).
2. Shifting in the time domain
For the signal x(n n
0
), we have:
Z
Z

Transform
Transform
Example
Consider the difference equation
x(n 1) 2x(n 2) = y(n)+ y(n + 1)
and represent its in z domain.
Solution:
Z
Z

Transform
Transform
3. Multiplication by complex exponential sequence
4. Convolution
If we have y(n) = x(n) h(n) than follows:
Z
Z

Transform
Transform
Example
By using z-transform find convolution of the
signals:
x(n) = u(n) and h(n) = (1/3)
n
u(n)
Solution
By using property 3, we obtain:
Z
Z

Transform
Transform
Now, we have:
The region of convergence is |z| >1. y(n) will
be obtain by using inverse z-transform. First we
will write previous equation in the following form:
where B = 3/2 and C = 1/2.
Thus we can write:
Z
Z

Transform
Transform
Relationship between z-transform, Fourier
transform and discrete Fourier transform
If we compare definition of the Fourier transform
of discrete signals and z-transform definition we
see that Fourier transform is equal to the z-
transform for |z| = 1. Thus the values of the z-
transform on the |z| = 1 in z domain are the values
of the Fourier transform of the sequence.
By expressing the complex variable z in polar form
as z = re
j
, we obtain:
Z
Z

Transform
Transform
taking r = 1 follows:
In general case the z-transform on the circuits
defined by r is equal to the Fourier transform of
the sequence x(n) multiplied by rn.This is
reason why the z-transform exist in the some cases
when the Fourier transform does not exist.
Z
Z

Transform
Transform
One example that confirm the previous
statement is x(n) = u(n).
The Fourier transform of this sequence does not
converge, but the z-transform converge for r > 1.
From the previous considerations we know that
the values of discrete Fourier transform are the
samples of the Fourier transform of discrete
signals.
This means that the values of
discrete Fourier transform are equal to the samples
of the z-transform for |z| = 1.
Z
Z

Transform
Transform
Example
Find the z-transform of the sequence:
x(n) = u(n) u(n 4)
and in the case N = 8 find the Fourier transform
and the discrete Fourier transform by using result
obtained for z-transform.
Solution:
Z
Z

Transform
Transform
System function
Consider a system where is:
y(n) = x(n) h(n)
Having in mind properties of the z-transform
follows:
Y(z) = X(z)H(z)
Z
Z

Transform
Transform
The z-transform of the impulse response is referred
to as the system function.
The system function evaluated on the unit circle
(|z| = 1) is the frequency impulse response of the
system.
From the previous considerations we know that
stable system must satisfied condition:
Z
Z

Transform
Transform
Consider now the z-transform of the h(n):
From the previous equation follows that in the
case of stabile systems unit circle |z| = 1
must belong to the region of convergence of the
function H(z).
For causal system the region convergence must be
exterior of a circle passing through the pole of
H(z) that is farthest from the origin.
Z
Z

Transform
Transform
Example
Check the causality of the system:
Solution
We see that the region of convergence is |z| > 1/2.
Thus the system is causal.
Z
Z

Transform
Transform
Consider now the system described by a linear
Constant coefficient difference equation, i.e.
the system that satisfy the general N th order
difference equation:
Applying the z-transform to each side of
previous equation, we have:
Z
Z

Transform
Transform
where the property of the z-transform:
is used.
Now, we can write:
In the case where A
j
= 0 for j > 0, the system
with finite impulse response is obtained (FIR) and
in that case we have:
Z
Z

Transform
Transform
Example
Find the impulse response of the causal system
described by:
and check its stability.
Solution:
Z
Z

Transform
Transform
Poles of this function are: z
1
= 1 and z
1
= 1/4.
Since the system is causal the region of
convergence is |z| > 4. This means that the
system is not stabile (unit circle does not belong to
the region of convergence).
In order to determinate h(n), write H(z) in the form:
Z
Z

Transform
Transform
Examples:
1. Find the z-transform of the sequence
x(n) =(n 5).
Solution:
2. If X(z) is the z-transform of x(n), find the z-
transform of:
Z
Z

Transform
Transform
By substitution n + k = m, we have:
Y(z) = X(z)X(1/z)
3. Find the impulse response of the system with z-
transform:
Solution:
Having in mind expansion in the series:
we can write:
Z
Z

Transform
Transform
Thus,
4. Find the causal sequence x(n), if its z-transform
has the form:
Solution:
Write X(z) in the form:
Z
Z

Transform
Transform
Thus we have:
x(n) =[ 1.25 0.25(0.2)
n
]u(n)
Z
Z

Transform
Transform
5. For the system shown in Figure, find system
function, check stability, and determine response
on the signal
x(n) = (n) 2 (n 1).
Solution:
From the Figure we have:
Z
Z

Transform
Transform
The system function is:
The pole of this system is z = 2, this fact means if
the system is causal it is not stabile.
If x(n) = (n) 2 (n 1) than:
ESTIMATION THEORY
Introduction to random signals
At the beginning, we will give some important
definitions:
Mean of the process is defined as:
The operator of mean value E is linear, i.e.:
If the random variables are independent or
uncorrelated then:
ESTIMATION THEORY
A sufficient condition for independence is:
In this case the random variables are statistically
independent.
Mean square value of x(n) is:
ESTIMATION THEORY
Correlations and covariances
The autocorrelation is defined as:
or,
where denotes complex conjugation.
The cross-correlation of two random processes x(n)
and y(n) is defined as:
ESTIMATION THEORY
The autocovariance is defined as:
If n = m the variance is obtained:
In the case of stationary process, the variance is
independent of time and denoted as
In the case of random processes that are
stationary in the wide sense we have:
ESTIMATION THEORY
In this case autocorrelation depends only on the
time difference m n, thus:
Also we have:
ESTIMATION THEORY
White noise
The signal that has the autocorrelation in the form:
is called white noise.
The name comes from the fact that the Fourier
transform of this is constant
It means that power density spectrum of this
function is constant, what is the property of the
white light.
In the case of real noise w we have:
ESTIMATION THEORY
Power density spectrum
Consider the z-transform of the autocorrelation
r
zz
(n) (in the case of stationary signals):
Define now S
xx
() as values of the z-transform
on the unit circle:
Note that S
xx
() is a real-valued function, since
ESTIMATION THEORY
From the above equation we
can write:
Having in mind the definition of r
xx
(n), we have:
Thus, the expected signal power is equal to the
integral of S
xx
() . This is the reason why S
xx
() is
called power spectral density. Later, it will be
shown that the signal energy within the frequency
region [
1
,
2
] is equal to the integral of S
xx
()
from
1
to
2.
ESTIMATION THEORY
Linear systems and random signals
For a linear system we know that:
If the signal x(n) is stationary, i.e. E{x(n k)}= MI
x
,
we can write:
ESTIMATION THEORY
The auto-correlation of the output signal is
defined as:
In the case of stationary signal when
r
xx
(n i,m k)= r
xx
(n m k i), we have:
We can conclude,if the signal x(n) is stationary
in the wide sense, then the signal at the output of
the linear system is stationary in the wide sense,as
well.
ESTIMATION THEORY
Find, now, the z-transform of the r
yy
(n,m).
We have:
By substitution l=p k + i, we obtain:
If h(n) is real:
ESTIMATION THEORY
Power spectral density is:
Therefore if |H(e
j
)|2 is an ideal band-pass
filter for the interval [
1
,
2
] then the expected
power of the output signal is:
since S
xx
(
1
) could be considered as a constant
within [
1
,
2
] for small
2

1
. This proves
that S
xx
(
1
) is the spectral power density.
ESTIMATION THEORY
Optimal filtering
Consider the signal in the form:
x(n) = s(n) + w(n)
where s(n) is desired signal and w(n) is the noise.
If we assume that the signal and noise are
uncorrelated we can write:
ESTIMATION THEORY
For the case when the power spectral density of
the signal and noise are not overlapped, we
can easily obtain denoised signal..
Namely, passing the signal through the
bandpass Filter which passes only the frequency
components of S
ss
(), denoised signal is obtained.
However, if the noise is white (existing in the
whole frequency range) by using previous method
it is possible to obtain only partial denoised signal.
In the general case the problem is in determination
of d(n) = s(n + m) in the most accurate
way.
ESTIMATION THEORY
If m = 0 we have the case of optimal filtering.
In some cases, it is necessary to predict the values
of the signal in the future, then m > 0.
However, in some cases we need to determine
some previous value of the signal. In this case we
have optimal smoothing, and m < 0.
Processing by using IIR system
Consider an IIR system defined by:
ESTIMATION THEORY
The mean square error is:
From this we have:
Define, now, correlation functions:
From the above equations we have:
If the signal and noise are uncorrelated, we have:
ESTIMATION THEORY
Fourier domain form of the optimal filter, when
d(n) s(n), is:
ESTIMATION THEORY
Power spectrum estimation
The mean value of the n th sample of the
sequence x(n) can be estimated by:
where x
i
(n) is the n th sample in the i th
measurement.
The special class of the random processes are
ergodic processes. In this case probability
averages are equal to time average, i.e.
ESTIMATION THEORY
The process is ergodic if we can estimate its
statistically properties on the base of only one
random signal.
In previous equation
x
is random value
because we have finite number of samples 2N + 1.
In the case N the value of
x
will be
sufficiently accurate.
Estimate, now, autocorrelation function (which
is the mean value of the product
x(n + m)x

(n)):
ESTIMATION THEORY
In the case of stationary process we have:
If we know x(n) only within the interval
N n N, then x(n + m) will be known only for
n N m for positive number m, and
n N |m| for negative number m.
If we want to avoid calculations for positive and
negative value of m, we will introduce symmetric
product:
ESTIMATION THEORY
In this sum we have 2N + 1 |m| terms, but we
average it with 2N + 1. This is the reason why
we have systematic error that can be avoided by:
Thus is the biased estimate of the
autocorrelation
ESTIMATION THEORY
Definition and variance of the Periodogram
Define the Fourier transform of the biased
autocorrelation function:
Since
it can be shown that:
ESTIMATION THEORY
The spectrum estimate I
N
() is called the
periodogram. The expected value of the
periodogramis:
Taking
ESTIMATION THEORY
Thus, the periodogramis a biased estimate of
the power spectrum S
xx
(). The previous
equation can be written, using convolution, in the
form:
where W
B
is the Fourier transform of the so called
Bartlett window ( for
|m| < 2N 1) given by:
ESTIMATION THEORY
Variance of the Periodogram
Express the periodogramin the form:
The covariance at frequencies
1
and
2
of I
N
() is:
ESTIMATION THEORY
In the case of white Gaussian process we have:
Thus,
ESTIMATION THEORY
Therefore, we have:
The variance is:
ESTIMATION THEORY
Smoothed spectrum estimators
If the sequence x(n), 0 n N 1, is divided
into K segments of M samples, the periodogram
will be:
If we assume that periodograms are independent
of one another, we have
ESTIMATION THEORY
By assumption that K periodograms are
statistically independent, then B
xx
() is the
mean of the set of K independent observations of
the periodogramI
M
() :
From previous equation it is clear that as K
becomes large, the variance approaches zero, so
this smoothed estimate is a consistent estimate.
ESTIMATION THEORY
EFFECTS OF FINITE REGISTER LENGTH
Signal values in digital signal processing are
stored in a binary format, using registers with
a finite length. This can cause the error.
Namely, if we have number with b bits
multiplied by another one with b bits, the result
will be data with 2b bits. If the length of register is
less than2b we will have truncation error. This
error is:
where Q[x] and x are numbers after and before the
truncation.
ESTIMATION THEORY
If we consider, now, effects of quantizations of
analog signal, we know:
Every samples must be represent by finite
length number, so we will have truncation or
rounding to the nearest quantization level and it
will cause quantization error. This error can be
expressed by noise e(n), than we have:
where x(n) is exact value and e(n) quantization
error.
ESTIMATION THEORY
In the case of rounding the errors is in the range:
/2 e(n) /2
while in the case of truncation it is:
e(n) 0,
where is quantization width = 2
b
.
If we want to give a model to describe the effects
of quantization we will assume:
1. The sequences of error samples {e(n)} is a
sample sequence of stationary random process.
2. The error sequence is uncorrelated with the
sequence of exact samples {x(n)} .
ESTIMATION THEORY
3. The error is a white-noise process.
4. The probability distribution of the error process
is uniform over the range of quantization
error.
Find signal to noise ratio in the case of rounding.
According assumption 4, we have that probability
distribution p
en
(e) = 1/.
Thus:
ESTIMATION THEORY
Now we have:
Multidimensional discrete signals and systems
Discrete N-dimensional signal can be defined as:
where n
1
, n
2
,.....,n
N
are integers.
Multidimensional discrete signals and systems
By analogy with the one-dimensional case we can
define:
1. Unite impulse:
2. Unite step:
Multidimensional discrete signals and systems
3. Complex exponential series
Discrete multidimensional system can be defined
by:
with x(n) and y(n) are defined input and output
signal, respectively.
Multidimensional discrete signals and systems
System is linear if:
If we denote multidimensional unite impulse
response with:
Where . The previous equation is
N-dimensional convolution denoted by:
Multidimensional discrete signals and systems
Causality and stability are defined in full analogy
with the one-dimensional case.
Fourier transform of N-dimensional discrete signals
The Fourier transform of an N-dimensional
discrete signal is defined by:
The inverse Fourier transform is given by:
Multidimensional discrete signals and systems
In the case of two-dimensional signal we have:
Example:
Find the Fourier transform of the signal:
Multidimensional discrete signals and systems
Solution:
Also, by analogy with the one dimensional
sampling theorem, it is easy to show that:
Multidimensional discrete signals and systems
where it has been assumed:
Multidimensional discrete signals and systems
Multidimensional discrete Fourier transform and
FFT algorithms
Consider two-dimensional discrete Fourier
transform: the simplest 2D FFT algorithm are
based on the FFT algorithm for one-dimensional
case. Namely:
We see that for a fixed value n1, the second
sum presents one-dimensional discrete Fourier
transform which can be calculated by using some
of the FFT algorithms.
Multidimensional discrete signals and systems
Thus:
This procedure should be repeated for all n1.
Two-dimensional discrete Fourier transform
will be obtained as:
Calculations should be performed for all k
2
.
Multidimensional discrete signals and systems
Ratio of number of additions and summations
for discrete Fourier transform by using
definition and FFT algorithm is given by:
In the case of M = 128 this ratio is 1170 ( if need
one second with the FFT than 19,5 minutes would
be needed by using calculation based on the
definition).
Multidimensional discrete signals and systems
Radon transform and computers Tomography
Integral along line AB is:
where AB is defined by:
The previous integral can be written in the form:
Multidimensional discrete signals and systems
The previous integral can be written in the form:
Previous integral defines projection of function
f(x, y) with respect to variable t for an arbitrary
angle .
Is it possible to reconstruct function f(x, y) on
the base projections?
Answer is yes.
Multidimensional discrete signals and systems
Proof:
Consider the Fourier transform F(u, v) of the
function f(x, y):
Multidimensional discrete signals and systems
The Fourier transform of a projection is:
Consider as a special case the value of the F(u, v),
along the line v = 0, then we have:
Multidimensional discrete signals and systems
Thus, we have obtained that the Fourier
transform of the function f(x, y) along axis v = 0
is equal to the Fourier transform of projection for
the angle = 0.
This result can be generalized. It can be shown
that the Fourier transform of f(x, y) along an
arbitrary line defined by angle with respect to u
axis is equal to the Fourier transform of the
projection defined by angle with respect to x
axis.
Multidimensional discrete signals and systems
The previous claim can be proved. Denote with
f(s, t) the function f(x, y) rotated in the coordinate
system. Relationship between variables
(x, y) and (s, t) is:
Since:
The Fourier transform of the projection is:
Multidimensional discrete signals and systems
In x, y coordinate system we obtain:
Thus, the previous claim is proved.
Multidimensional discrete signals and systems
Finally, we can conclude:
Function f(x, y) can be obtained on the following
way:
1. Find the projection for 0 .
2. Determine the Fourier transforms of the
projections which give the Fourier transform of
the function f(x, y).
3. Compute the inverse Fourier transform and it if
function f(x, y).
Multidimensional discrete signals and systems
Note that the Fourier transform of function
f(x,y) will be obtained in polar raster. If we
want to use FFT algorithms it is necessary to have
the Fourier transform in rectangular raster.
One possible solution is interpolation of values
from the polar to the values on the rectangular
raster.