FinalProject_Complete_6

Signal detection Theory Final Project Report:
Spectral Analysis of Nonuniformly Sampled Data:
A New Approach Versus the Periodogram
Petre Stoica, Fellow, IEEE, Jian Li, Fellow, IEEE, and Hao He, Student Member, IEEE
Farhad Gholami
1

Abstract:
Power Spectral Density (PSD) for a random signal y(t) is defined as expected value (average)of power
of y(t) and is important when analyzing random processes. In practice we need estimate PSD from a
limited number of samples which are noisy using periodogram.
We will see why periodograms generally suffer from two drawbacks which are:
1)Poor resolution due to local leakage through the main lobe of the spectral window.
2)Significant global leakage through the side lobes
First we review PSD s and will explain periodograms and why least-squares periodogram (LSP) is
preferable to the Fourier periodogram from a data-fitting point of view and also it is not
computationally very complicated.
To solve these issues new method proposed in this paper, which can be interpreted as an iteratively
weighted LSP that makes use of a data-dependent weighting matrix built from the most recent spectral
estimate.
Because this method was derived for the case of real data (which is more complicated to deal with in
spectral analysis than the complex data), it is iterative and it makes use of an adaptive ( data-dependent)
weighting, we referred to it as the real-valued iterative adaptive approach (RIAA).
Power Spectral density(PSD) and problem definition:
Considering formal definition of PSD :
We assume samples {x1, . ..,xN} need to be very large which creates below practical problems:
1)We are only given one sequence so can do expected values.
2)We have limited number of samples so can not let N becomes close to infinite.
So we want a method to determine estimate of PSD using a finite number of samples.
Applications of Spectral Estimation:
Manyof systems dealing with random processes for practical reasons need to estimate PSD .
Speech: Formant estimation (for speech recognition) , Speech coding or compression
Radar and Sonar: Source localization with sensor arrays, Synthetic aperture radar imaging and feature
extraction
Electromagnetics: Resonant frequencies of a cavity
2

Communications:Code-timing estimation in DS-CDMA systems
Spectral Density Estimation Techniques:
Parametric Methods: Assume underlying stationary stochastic process has a certain structure which
can be described using a small number of parameters (for example, using moving average model).
Task: Estimate the parameters of the model that describes the random process.
Nonparametric Methods: Estimate spectrum of the process without assuming that the process has
any particular structure.( for example Periodogram , Least-squares spectral periodogram, based on
least squares fitting to known frequencies)
Trade-Offs: (Robustness vs. Accuracy): Parametric Methods may offer better estimates if data
closely agrees with assumed model. Otherwise, Nonparametric Methods may be better
Periodogram Definition (derived from PSD definition):
We define periodogram for provided samples are {y1, . ..,yN} to estimate power spectral density as:
Which is derived from PSD definition by omitting expected value and limiting number of samples to
N.
Periodogram Variance:
Periodoram estimation of PSD is often noisy, one way of noise reduction is averaging as descibed
below:
3

Windowing effect on periodogram:
The Periodogram can be interpreted by DFT multiplied by a window in time domain which is a
convolution with sinc likes in frequency domain:
For a rectangular window, in frequency domain we will have:
This window effect creates two main problem for our estimation as we describe them as:
1)Local leakage.
2)Gobal leakage.
4

Local leakage is due to the width of the main beam of the spectral window, and it is what limits the
resolution capability of the periodogram.
Global leakage is due to the side-lobes of the spectral window, and is what causes spurious peaks to
occur (which leads to “false alarms”) and small peaks to drown in the leakage from large peaks (which
leads to “misses”).
The Modified Periodogram:
Considering that rectangular window among all windows with the same width N , has the narrowest
main lobe (a good quality here) but also has big side lobes.
By replacing WR[n] by a different window we can have lower side lobes but wider main lobes.
Examples are Bartlett and Hamming windows. To reduce variance we can perform local averaging of
the periodogram which in turn can increase bias:
5

Least Squares (LS) optimization:
Modeling periodic behavior in a noisy time series suggest to use a method which can eliminate noise
effect from data. Least-squares spectral analysis (LSSA) estimating a frequency spectrum, based on a
least squares fit of sinusoids to data samples which is more immune to noisy data.
We can express the Periodogram as the solution of the Least Squares (LS) optimization fitting problem
Px will be obtained by solving the Least Square problem using sudeo inverse matrix:
Fourier Periodogram:
The Fourier transform periodogram (FP) associated with N samples y(tn) is given by:
It can be verified that PF can be obtained from the solution to the following least-squares (LS) data
fitting problem:
6

Because samples are real values, the Least Square criterion above can be rewritten as:
We can minimize the first terms (sinusoidal data assumption)but second term has no data fitting
interpretation and only acts as an additive data-independent.
Least Square Periodogram:
As we saw in the case of real (sinusoidal) data, considered in this paper, the use of FP is not completely
suitable, and that a more satisfactory spectral estimate should be obtained by solving the following LS
fitting problem:
By omitting the dependence of and on , for notational simplicity). Using and (5)
We can re- parameterize the LS criterion as :
The solution to the minimization problem is well known :
The power of the sinusoidal component with frequency , corresponding to estimations os a and b , is
given by:
7

The LS periodogram is accordingly given by:
RIAA:
The new method (RIAA) can be interpreted as an iteratively weighted LSP that makes use of a data-
dependent (adaptive) weighting matrix built from the most recent spectral estimate(real valued data).
The amplitude and phase estimation (APES) method for uniformly sampled data, has significantly less
leakage ( local and global) than the periodogram.
Here we extend APES to the non-uniformly sampled data case(RIAA).
Paper presents a procedure for obtaining a parametric spectral estimate, from the nonparametric
estimate, by means of a Bayesian information criterion (BIC).
Both LSP and RIAA provide nonparametric spectral estimates in the form of an estimated
periodogram. We use the frequencies and amplitude corresponding to the dominant peaks of (first the
largest one,...) in a Bayesian information criterion (BIC), to decide which peaks we should retain and
which ones we can discard.
The use of BIC for the said purpose can be viewed as a way of testing the significance of the dominant
peaks of the periodograms.
Using this notation, we can rewrite the LS fitting criterion in the following vector form :
8

Assuming that Qk is available, and is invertible, it would make sense to consider the following
weighted LS (WLS) criterion:
Indeed, it is well known that the estimate of θ obtained by minimizing last equation is more
accurate, under quite general conditions and is given by:
Weighted Least Square (WLS) periodogram can be defined as:
The PWLS estimate require inversion of a matrix , this would be computationally intensive task.
To reduce the computational complexity , we define a new matrix:
And we will have:
and:
Which gives us:
which is computationally simpler and matrix inverse needs to be computed only once for all values of
k=1,...K
Here we explain how to resolve the problem that Γ depends on the θ quantities that we want to
estimate, and consequently that can not be implemented directly.
The only apparent solution to this problem is an iterative process, the proposed RIAA algorithm below
will address this issue.
9

In most applications, the RIAA algorithm is expected to require no more than 10–20 iterations .
RIAA Performance:
Here we provide some insights into expected behavior of RIAA. In particular, we explain intuitively
why RIAA is expected to have less (both local and global) leakage than LSP.
RIAA estimates residual matrix Qk using a theoretical formula , along with the most recent spectral
estimate available; in the spectral analysis problem considered in this paper, we dispose of only one
realization of y.
Now let define Hk define a matrix as solution to the following constrained minimization problem:
where f is a monotonically increasing function on the domain of positive definite matrices. We use
Hk to obtain an estimate θ :
Now we show the solution to above minimization problem is given by:
10

We can write this in the following form:
which is equivalent to:
Because the matrix in the left-hand side of above is positive semi-definite, the proof that Hk is the
solution to WLS estimate is concluded.
The intuition to how WLS estimate reduces leakes is interesting.
The Hk matrix that solves optimization problem can be viewed as a “filter” that passes the sinusoidal
component of current interest (with frequency ω ) without any distortion and attenuates all the
other components in as much as possible.
To illustrate this property of Hk, we use the fact that, by assumption, the data contains a finite (usually
small) number of sinusoidal components. This means that there are only a limited number of
frequencies which contribute significant terms to Qk .
Let Wp be one of these frequencies. Then Hk should be nearly orthogonal to Ap , which means that Hk
acts as a filter for any strong sinusoidal component in whose frequency is different from Wk. This
observation explains why RIAA can be expected to have significantly reduced leakage problems
compared with LSP.
Bayesian Information Criterion (BIC):
The Bayesian Information Criterion (BIC) rule is an statistical tests of hypothesis testing to decide if
most dominant peak of LSP is significant.
To explain how this can be done, we sort “frequencies, amplitude and phase” related parameters
corresponding to the largest peaks denote the values taken by either the LSP or the RIAA periodogram
at the points of the frequency grid as shown below (M largest peaks) :
11

Under assumptions that the data sequence consists of a finite number of sinusoidal components and of
normal white noise, and these values are the maximum likelihood (ML)of “frequency , amplitude and
phase”, the BIC rule, estimates M as:
BIC is made of two terms:
1)LS data fitting term that decreases as M increases.
2)Complexity penalization term which increases with increasing M.
Therefore BIC estimate is a tradeoff between in-sample fitting accuracy and complexity of the
sinusoidal data description.
Sampling pattern design
Method for designing an optimal sampling pattern that minimizes an objective function based on the
spectral window.
1)Assume that a sufficient number of observations are already available, from which we can get a
reasonably accurate spectral estimate.
2)Make use of this spectral estimate to design the sampling times when future measurements should be
performed (objective Function optimization).
Conclusion:
LSP and RIAA are nonparametric methods that can be used for the spectral analysis of general data
sequences with both continuous and discrete spectra. However, they are most suitable for data
sequences with discrete spectra (i.e., sinusoidal data) which is the case we emphasized in this paper.
For the latter type of data, we presented a procedure for obtaining a parametric spectral estimate, from
the LSP or RIAA nonparametric estimate, by means of a Bayesian information criterion (BIC).
The use of BIC for the said purpose can be viewed as a way of testing the significance of the dominant
12

peaks of the LS or RIAA periodograms, a problem for which there was hardly any satisfactory solution
available
We also discussed a possible strategy for designing the sampling pattern of future measurements, based
on the spectral estimate obtained from the already available observations.
Appendix: Matlab simulation for periodogram:
%=============================================================
% program pdf_estimate_test.m
%=============================================================
age=[0:600]';
rand('state',0);
ager=age+0.3*rand(size(age))-.15;
ager(1)=age(1);
ager(601)=age(601);
depth=age/10; %creates depth between 0 and 60
bkg=interp1([0:10:600],rand(61,1),ager);
f1=1/95;
f2=1/125;
sig=cos(2*pi*f1*ager)+cos(2*pi*f2*ager+pi); %??
o18=sig+bkg;
%pick frequencies to evaluate spectral power
freq=[0:0.0001:0.02]';
power=lomb(age,o18,freq);
power(1)=0;
%normalize to average 1
power = power/std(power);
%plot the results
figure;
plot(freq,power);
xlabel('frequency(cycle/kyr)');
ylabel('spectral power');
%=============================================================
% program pdf_estimate_test.m
% This program should be saved with the name pdf_estimate.m
% Calculates periodogram
%=============================================================
function power = pdf_estimate(t,y,freq)
%set constants
nfreq=length(freq);
fmax=freq(nfreq);
fmin=freq(1);
power=zeros(nfreq,1);
13

f4pi=freq*4.*pi;
pi2=pi*2.;
n=length(y);
cosarg=zeros(n,1);
sinarg=zeros(n,1);
argu=zeros(n,1);
var=cov(y); %variance
%subtract mean
yn=y-mean(y);
%do one Lomb loop(the t loop is vectorized)
nfreq
for fi=1:nfreq
sinsum=sum(sin(f4pi(fi)*t));
cossum=sum(cos(f4pi(fi)*t));
tau=atan2(sinsum,cossum);
argu=pi2*freq(fi)*(t-tau);
cosarg=cos(argu);
cfi=sum(yn.*cosarg);
cfi
cosnorm=sum(cosarg.*cosarg);
sinarg=sin(argu);
sfi=sum(yn.*sinarg);
sinnorm=sum(sinarg.*sinarg);
power(fi)=(cfi*cfi/cosnorm+sfi*sfi/sinnorm)/(2*var);
end;
14

References
1) Spectral Analysis of Nonuniformly Sampled Data: Petre Stoica , Jian-Li, Hao-Ha
2) J. Li and P. Stoica, “An adaptive filtering approach to spectral estimation
3) EEL 6537 { Spectral Estimation Jian Li ,Department of Electrical and Computer
Engineering University of Florida
15

FinalProject_Complete_6

More Related Content

What's hot (17)

Viewers also liked (16)

Similar to FinalProject_Complete_6 (20)

More from Farhad Gholami (7)

FinalProject_Complete_6