Lecture 4

Spectrum Estimation
Power Spectral Density
PSD

Wiener Khinchin Theorem

 Problem with PSD Estimation

Both of these approaches are called nonparametric classical

approaches. They try to do the best with the available data.
The only assumption is that the underlying process is
wide sense stationary (WSS).
‘Modern’ Parametric Approach
Some useful concepts from statistics

Each PSD estimate is a realization of the random process

 Some useful concepts from statistics(cont)

For each realization of x[n] we get a different value for the estimate of A
Some useful concepts from statistics(cont)
Some useful concepts from statistics(cont)
Nonparametric Spectral Estimation
 Family of Non Parametric Methods

Classical Methods
(Fourier Transform Nonclassical Methods
Based) (Non Fourier Based)

Filterbank Approach
.Minimum Variance
Periodogram Based
.Periodogram ACF Based
.Modified Per. .Blackmen Tukey
.Bartlett
.Welch
 Family of Classical Methods

 1 2


M
S x (w)  lim E   x (n)e  jwn  S x ( w)  rx (k )e  jwk
 2 M  1 n  M
M 
 k 

Periodogram
Blackman-Tukey

Modified Periodograms Bartlett's Method

(Use of Windows) (Average Periodograms)

Welch's Method
(Average Windowed Periodograms)
Periodogram
 Periodogram Definition
Based on the definition of PSD:
2
 1 M 
S x ( w)  lim E   x(n)e  jwn 
 2 M  1 n  M
M 


In practice we have one set of finite duration data.

This approach has 2 practical problems
1. Can’t do the expected value
2. Can’t do the limit

The periodogram is a method that ignores them both

2
ˆ 1 N 1
S PER ( w) 
N
 x
n 0
( n ) e  jwn

In practice we compute it using the discrete Fourier transform (DFT)

(possibly with zero padding) which computes the discrete time
Fourier transform (DTFT) at discrete frequency points
 Periodogram Computation
In practice the periodogram is computed using the
DFT (FFT) usually with zero padding. Thus computing
the Discrete Time Fourier Transform (DTFT) at discrete
frequency points.
2 2

ˆ 1 N 1 j kn 2
S PER (k ) 
N
 x ( n)e
n 0
Nz
; wk =
Nz
k

N-number of signal samples

Nz- DFT size with zero padding
Periodogram-Viewed as a filterbank

Although the Periodogram is Always implemented using

the DFT, a filterbank interpretation is also very helpful.

Defining the impulse response of an FIR filter as

 1 jnwi
 e , 0  n <N
hi (n)   N
0, ow
The frequency response of this filter then becomes
N 1
H ( w)   hi (n)e  jwn
n 0

 jn ( w  wi )( N 1) / 2 sin( N ( w  wi ) / 2)
e
N sin(( w  wi ) / 2)
Periodogram-Viewed as a filterbank (cont)
With this definition of hi(n), the output of the ith filter becomes
n
yi (n)  x(n) * hi (n)  
k  n  N 1
x(k )hi (n  k )

1 n

N

k  n  N 1
x(k )e j ( n  k ) wi

2
Note that if one chooses n=N-1 the term yi ( N  1) gives the
periodogram
2 2
1 N 1
j ( N 1) wi 2 1 N 1
 
2 j ( N 1 k ) wi  jkwi
yi ( N  1)  x ( k ) e e x ( k ) e
N k 0 N k 0
2
1 N 1
  x ( k ) e  jkwi
 NSˆPER ( w)
N k 0
 Performance of the Periodogram
For a good PSD estimate one should at least have:

N 
 
lim E Sˆx ( w)  S x ( w)
This is the asymptotically unbiased condition.
(Normally one prefers it to be unbiased for finite N)


lim var Sˆx ( w)  0
N 

It is important to investigate the periodogram to find
out if these characteristics are possessed
Performance of the Periodogram (BIAS)
The periodogram is asymptotically UNBIASED.

Taking the expected value of the Periodogram gives

2
1 
 N 1 


E SˆPER ( w)   E   x (n)e  jnw
N 

 n 0 
 Note that rx(n-m) is
1   N 1  jnw  
N 1
jmw  
constant on each diagonal.
 E    x ( n )e    *
x ( m )e  The double sum is doing a
N   n0   m 0  sum on diagonals.
1 N 1 N 1
  x
N n 0 m 0
r ( n  m )e  jw ( n  m )

N 1
 k 
  1  rx ( k )e
 jwk
Bartlett triangular window
k  ( N 1)  N
1 As N tends to infinity the expected value
 S x ( w) *WB ( w)  S x ( w) of the periodogram estimate tends to the
2 True PSD. Thus the periodogram is
asymptotically unbiased
Performance of the Periodogram (Variance)
The variance of the periodogram DOES NOT
tend to zero as N tends to infinity.

The variance can be approximated as

Let us consider this term
The above analysis was carried out for white noise.

For non white processes

 Matlab Implementation
[Pxx,F] = PERIODOGRAM(X,WINDOW,NFFT,Fs)
returns a PSD computed as a function of physical frequency (Hz).
Fs is the sampling frequency specified in Hz. Fs is empty, it defaults to 1 Hz.
Power Spectral Density Estimate via Periodogram
-10

-15
example:
-20
Fs = 1000; t= 0:1/Fs:.3;
x = cos(2*pi*t*200)+
Power/frequency (dB/Hz)

-25 randn(size(t));
periodogram(x,[],'twosided
-30

-35
N=300
-40
Nz=512
-45

-50
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Frequency (kHz)
example:
Fs = 1000; t= 0:1/Fs:1; x = cos(2*pi*t*200)+ randn(size(t));
periodogram(x,[],'twosided',1024,Fs);

Power Spectral Density Estimate via Periodogram

0

-10

N=1000
-20
Nz=1024
Power/frequency (dB/Hz)

-30

-40

-50

-60

-70
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Frequency (kHz)
 example:
Fs = 1000; t= 0:1/Fs:10; x = cos(2*pi*t*200)+ randn(size(t));
periodogram(x,[],'twosided',1024,Fs);

Power Spectral Density Estimate via Periodogram

10

-10
N=10000
Power/frequency (dB/Hz)

-20 Nz=10200
-30

-40

-50

-60

-70

-80
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Frequency (kHz)
 example:
Fs = 1000; t= 0:1/Fs:100; x = cos(2*pi*t*200)+ randn(size(t));
periodogram(x,[],'twosided',1024,Fs);

Power Spectral Density Estimate via Periodogram

20

N=100000
Power/frequency (dB/Hz)

-20
Nz=100200
-40

-60
Spectral estimation
of the deterministic
-80
component improves
with increasing N
-100
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Frequency (kHz)
Example: Estimating the PSD of a cos at 200Hz buried in noise
x (t) = cos(2π200t) + n(t)

Matlab:
for kk=1:4;
randn('state',0);
Fs = 1000;
t = 0:1/Fs:10^(kk-2);
N=length(t);Nz=N+10;
x = 1*cos(2*pi*t*200) + 1*randn(size(t));
[Pxx,f]=periodogram(x,[],'twosided',Nz,Fs) ;
figure(1);
subplot(2,2,kk),plot(f,((Pxx)))
figure(2)
subplot(2,2,kk),periodogram(x,[],'twosided',Nz,Fs) ;end
0.025 0.25

0.02 0.2

0.015 0.15

0.01 0.1

0.005 0.05

0 0
0 500 1000 0 500 1000

2.5 25

2 20

1.5 15

1 10

0.5 5

0 0
0 500 1000 0 500 1000

No dB scale
 Power Spectral Density Estimate via Periodogram Power Spectral Density Estimate via Periodogram
-10 0

-10
-20
Power/frequency (dB/Hz)

Power/frequency (dB/Hz)
-20
-30
-30
-40
-40

-50
-50

-60 -60
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Frequency (kHz) Frequency (kHz)

Power Spectral Density Estimate via Periodogram Power Spectral Density Estimate via Periodogram
10 20

0
0
-10
Power/frequency (dB/Hz)

Power/frequency (dB/Hz) -20

-20

-30 -40

-40
-60
-50
-80
-60

-70 -100
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Frequency (kHz) Frequency (kHz)
 Filtering White noise through an FIR Filter

for kk=1:4
Fs = 1000; t = 0:1/Fs:10^(kk-2);
N=length(t);Nz=N;
x = randn(size(t));
h=[1 2 1]/4;y=filter(h,1,x); %This is an FIR filter
[Pxx,f]=periodogram(x,[],'twosided',Nz,Fs) ;
%[Pxx,f] = periodogram(x,window,nfft,fs)
[Pyy,f]=periodogram(y,[],'twosided',Nz,Fs) ;
H = FREQZ(h,1,Fs*[0:Nz-1]/Nz,Fs);
figure(1)subplot(2,2,kk),plot(f,(Fs*abs(Pxx)),'b')
hold on
plot(f,(Fs*abs(Pyy)),'r')
hold on
plot(f,abs(H).^2/1,'g')
figure(2), subplot(2,2,kk), plot(f,(10*log10(Fs*abs(Pxx))),'b')
end
 N=100 N=1000
6 7

5 6

5
4
4
3
3
2
2

1 1

0 0
0 100 200 300 400 500 600 700 800 900 1000 0 100 200 300 400 500 600 700 800 900 1000

N=10000 N=100000
10 12

10
8

8
6
6
4
4

2
2

0 0
0 100 200 300 400 500 600 700 800 900 1000 0 100 200 300 400 500 600 700 800 900 1000
Filtering white noise with an AR(1) process
1
H ( z) 
1  0.5 z 1
7 15

5
10
4

3
5
2

0 0
0 100 200 300 400 500 600 700 800 900 1000 0 100 200 300 400 500 600 700 800 900 1000

25 30

25
20

20
15
15
10
10

5
5

0 0
0 100 200 300 400 500 600 700 800 900 1000 0 100 200 300 400 500 600 700 800 900 1000
In all examples the Periodogram does not
converge to the true PSD with increasing N
This is impossible
U=1 ;
for rectangular
window
Fundamantal trade of between variance and resolution
Matlab funtion
function Px = bart(x,nsect)
%BARTBartlett's method of periodogram averaging.%----
%USAGEPx = bart(x,nsect) %
%The spectrum of a process x is estimated using Bartlett's
%method of periodogrm averaging.%
%x : Input sequence
%nsect : Number of subsequences to be used in the average%
%The Bartlett estimate is returned in Px using a linear scale.%

L = floor(length(x)/nsect);
Px = 0; n1 = 1;
for i=1:nsect
Px = Px + per(x(n1:n1+L-1))/nsect;
n1 = n1 + L;
end;
 Matlab Implementation of Bartlett method

Filtered white noise

for kk=1:3 function [Px,f] = bart1(x,nsect,Nz,Fs)

Fs=1000; L = floor(length(x)/nsect);
nsect=[5 50 500]; Px = 0;
t = 0:1/Fs:10^(kk-1); n1 = 1;
N=length(t); for i=1:nsect
Nz=N; [Pxx,f]=periodogram(x(n1:n1+L-1),[],…
x = randn(size(t)); 'twosided',floor(Nz/nsect),Fs) ;
h=[1 2 1]/4; Px = Px + Pxx/nsect;
aa=[1 -.5]; n1 = n1 + L;
y=filter(h,1,x); %This is an FIR filter end;
%y=filter([1 2 1],1,x);
[Py f]=bart1(y,nsect(kk),Nz,Fs);
H = FREQZ(h,1,f,Fs);
figure(1)
subplot(1,3,kk), plot(f,(Fs*abs(Py)),'r')
hold on
plot(f,abs(H).^2,'g')
end
 Bartlett Method with filtered white noise
FIR
N=1000, K=5, L=200 N=10000, K=50, L=200 N=100000, K=500, L=200
2 1.4 1.4

1.2 1.2
1.5
1 1

0.8 0.8
1
0.6 0.6

0.4 0.4
0.5
0.2 0.2

0 0 0
0 200 400 600 800 1000 0 200 400 600 800 1000 0 200 400 600 800 1000

IRR
7 6 4. 5

4
6
5
3. 5
5
4 3

4 2. 5
3
3 2

2 1. 5
2
1
1
1
0. 5

0 0 0
0 500 1000 0 500 1000 0 500 1000
N=KL
Ability to resolve (see) two closely spaced sinusoids
Example:Resolution