COURSE SYLLABUS: EE583 - STATISTICAL SIGNAL PROCESSING

Instructor: Dr. Edgar Satorius

1. Introduction
This class meets 6:40 PM - 9:10 PM every Thursday evening beginning January 15, 2015 and
ending on April 30, 2015. The final exam for this course is on Thursday May 7, 2015 from 79:00 PM. Our class room will be Kaprielian Hall (KAP) 165. The grader will be TBD.
IMPORTANT dates are: Midterm exam (March 5) and the Final exam (May 7). These are
hard dates so make sure you have no conflicts with these dates.
Please note: During this course we will have no classes on:
Mar. 19

Spring recess

Prerequisites: EE483, EE441, EE503

The sections given below are an outline of the topics I hope to cover in this course. Section 4
is mainly a review of material on discrete random processes. To do well in this course, you
should be thoroughly familiar with this material. The course texts ('Digital Processing of
Random Signals: Theory and Methods' by Porat; Chapter 2 as well as 'Adaptive Filter Theory'
by Haykin; Chps. 2 and 3) have good reviews on discrete random processes. Other related
courses include: EE563 (Estimation Theory which covers Kalman filters); EE586L
(Advanced DSP Design Lab); EE667 (Array Signal Processing) and EE668 (VLSI
Processors).
2. Grading and Computers
Computer
Project(s)
30%

Midterm
(open book & notes)
30 %

Final
(open book & notes)
30%

Homework

10 %

Throughout the semester I will assign 5-6 homework sets plus two or three computer
projects. The computer projects will help you learn the course material by conducting
practical computer experiments on real world problems. Each project will focus on a
reasonably well defined problem so that you can concentrate on learning the techniques -- not
coming up with problems. The results of your studies should be well documented in a report
with computer printouts/plots (no source listings required) to justify your conclusions. In
doing the computer projects, you can use any computer language you wish; however, I
encourage you to consider Matlab especially if you have experience with it. Matlab is a
very simple and powerful language that is particularly suitable for programming signal
processing algorithms. It also has a very nice graphical display capability and includes a
convenient mechanism for incorporating on-line help into the system. If you do use Matlab,
make sure you have the Signal Processing toolbox.

If you do well on the homeworks and the projects, then you will be able to perform well in
the class. Do the homeworks on your own (although you are free to discuss the problems with
other classmates). Likewise with the computer projects: you can discuss them with others, but
write them yourself. The midterm and final exams will be open text/notes exams. The
midterm will include all material covered up to the midterm and the final will cover the
remainder of the course (I will clearly describe what topics will be covered in the midterm
and final exams prior to the exams).
Policy on late assignment submittals: I will allow late submittals provided you let me
know in advance via e-mail. However, once the solutions are posted (typically about a week
after the assignments are due), no submittals will be accepted or graded.
3. Office Hours
My office hours are 5:15-6:30 Thursdays in PHE 414. TV students may call me during this
time (213 740 7654), or arrange an appointment for Thursday evenings. I strongly encourage
you to make use of this time to discuss problems with the course material or any related
aspects of digital signal processing which interest you. If you can't reach me otherwise, my email address is: [email protected].
Questions related to the homework, projects, Matlab, etc. should initially be addressed to the
grader. The graders e-mail address is: TBD.
COURSE OUTLINE:

4. Discrete random processes (Class 1)

[Porat, Chapter 2.1-2.11; Haykin (Chp. 1)]
[1]
[2]
[3]

Random variables, random processes, filtered random processes.

Ensemble averages, correlation, covariance, power spectrum, cross power
spectrum.
Ergodicity, time averages, biased & unbiased estimators, consistent
estimators.

5. Classical Spectral Estimation (Class 2)

[Porat, Chapters 4.1-4.6; Haykin (Chp. 1.16)]
[1]
[2]
[3]

Periodogram, correlogram.
Smoothed periodogram (Bartlett&Welch).
Resolution/variance tradeoffs.

6. Parametric Models of Random Processes (Classes 2-3)

[Haykin (Chapters 1.5-1.10)]
[1]
[2]

AR/MA/ARMA models.
Relationship between model parameters.

[3]

Relationship between model parameters and autocorrelation sequence YuleWalker equations.

7. Autoregressive and ARMA spectral estimation (Classes 3-5)

[Porat (Chapters 6-7); Haykin (Chapters 1.5-1.10)]
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]

Properties of AR spectral estimates.

AR correlation function estimation techniques.
AR reflection estimation techniques.
AR least squares prediction estimation techniques.
AR spectral estimation.
AR model order selection.
Modified Yule-Walker equations for ARMA modelling.
Separation of AR and MA processes.

8. Other spectral estimation techniques (Classes 5-6)

[Porat (Chp. 9); Haykin (Chapters 1.5-1.10)]
[1]
[2]
[3]

Prony spectral estimation.

Signal and noise subspace techniques.
Maximum likelihood (Capon) spectral estimation..

9. Digital Wiener filtering (Classes 6-7)

[Haykin (Chp. 2)]
[1]
[2]
[3]
[4]

Wiener smoothing and prediction filters.

Application of Wiener smoothing to noise cancelling.
Application of Wiener prediction filters.
Constrained, linear MMSE filtering.

10. Midterm, Class 8: March 5, 2015

11. Least mean squares adaptive filter (Classes 9-12)
[Porat (Chp. 7.7); Haykin (Chps. 4-6, 12-14)]
[1]
[2]
[3]
[4]
[5]
[6]

LMS adaptive algorithm.

Properties of LMS adaptive filter.
Normalized forms.
Finite precision effects.
Kalman filtering.
Adaptive Beamforming.

12. Orthogonalized and least squares adaptive filters (Classes 12-13)

[Porat (Chp. 8); Haykin (Chps. 8-10, 16)]
[1]
[2]

Frequency domain adaptive filters.

Adaptive lattice filters.

[3]

Godard algorithm.

13. Blind adaptive filtering (Classes 13-15)

[Haykin (Chp. 17)]
[1]
[2]
[3]

Cost functions.
Higher-order statistics.
Independent component analysis.
REFERENCES:

14.1. Required
[1]
[2]
[3]

Class notes: I have prepared a large amount of supplementary class notes

that are required for the course. These will be available at: blackboard.usc.edu.
Digital Processing of Random Signals: Theory and Methods, Boaz Porat, Dover
Books on Electrical Engineering, First Ed. 2008, ISBN-13: 978-0486462981.
Adaptive Filter Theory, S. Haykin, Pearson Education, 5-th edition, 2013, ISBN13: 978-0132671453.

14.2. Recommended Reading

[1]
[2]

[3]
[4]
[5]

[6]
[7]

Fundamentals of Statistical Signal Processing: Estimation Theory, Steven M. Kay,

Prentice Hall, 1st edition, 1993, ISBN-13: 978-0133457117.
Fundamentals of Statistical Signal Processing, Volume III: Practical Algorithm
Development, Steven M. Kay, Prentice Hall, 1st edition, 2013, ISBN-13: 9780132808033.
Digital Spectral Analysis, S. Marple, Prentice-Hall, 1987.
Fundamentals of Adaptive Filtering, Ali H. Sayed, John Wiley, 2003.
Statistical and Adaptive Signal Processing: Spectral Estimation, Signal
Modeling, Adaptive Filtering and Array Processing, D. Manolakis, V. Ingle,
S. Kogan, McGraw Hill, 1999.
Blind Deconvolution, S. Haykin, ed., Prentice-Hall, 1994.
Adaptive Filtering Primer with MATLAB (Electrical Engineering Primer Series),
A. Poularikas, Z. Ramadan, CRC Press, 1st edition, 2006, ISBN-13: 9780849370434.

14.3. Background Material

[1]
[2]
[3]
[4]
[5]

Discrete-Time Signal Processing, A. Oppenheim and R. Schafer, Prentice

Hall, 1999 (Chps. 2 -- 2.10, 11 & App. A).
Probability, Random Variables and Stochastic Processes, A. Papoulis,
McGraw-Hill, 1991.
Time Series, D. Brillinger, Holt-Reinhart-Wilson, 1975.
Applied Linear Algebra, G. Strang, Academic Press, 1976; OR:
Applied Linear Algebra, Noble and Daniel, Prentice Hall, 1977.

[6]
[7]
[8]

Matrix Computations, G. Golub, C. Van Loan, Johns Hopkins University

Press, 1983.
Numerical Recipes: The Art of Scientific Programming, W. Press, B.
Flannery, S. Teukolsky, W. Vetterling, Cambridge University Press, 1993.
MATLAB Reference Guide: High-Performance Numeric Computation and
Visualization Software, The MathWorks, Inc., South Natick, MA, 1984-92.

15. Statement for Students with Disabilities

Any student requesting academic accommodations based on a disability is required to register
with Disability Services and Programs (DSP) each semester. A letter of verification for
approved accommodations can be obtained from DSP. Please be sure the letter is delivered to
me (or to TA) as early in the semester as possible. DSP is located in STU 301 and is open
8:30 a.m.5:00 p.m., Monday through Friday. The phone number for DSP is (213) 740-0776.
16. Statement on Academic Integrity
USC seeks to maintain an optimal learning environment. General principles of academic
honesty include the concept of respect for the intellectual property of others, the expectation
that individual work will be submitted unless otherwise allowed by an instructor, and the
obligations both to protect ones own academic work from misuse by others as well as to
avoid using anothers work as ones own. All students are expected to understand and abide
by these principles. Scampus, the Student Guidebook, contains the Student Conduct Code in
Section 11.00, while the recommended sanctions are located in Appendix A:
http://www.usc.edu/dept/publications/SCAMPUS/gov/. Students will be referred to the
Office of Student Judicial Affairs and Community Standards for further review, should there
be any suspicion of academic dishonesty. The Review process can be found at:
http://www.usc.edu/student-affairs/SJACS/.
17. Academic Conduct
Plagiarism presenting someone elses ideas as your own, either verbatim or
recast in your own words is a serious academic offense with serious
consequences. Please familiarize yourself with the discussion of plagiarism in
SCampus in Section 11, Behavior Violating University Standards
https://scampus.usc.edu/1100-behavior-violating-university-standards-andappropriatesanctions. Other forms of academic dishonesty are equally
unacceptable. See additional information in SCampus and university policies on
scientific misconduct, http://policy.usc.edu/scientific-misconduct.
Discrimination, sexual assault, and harassment are not tolerated by the
university. You are encouraged to report any incidents to the Office of Equity and
Diversity http://equity.usc.edu or to the Department of Public Safety
http://capsnet.usc.edu/department/department-public-safety/online-forms/contactus.
This is important for the safety of the whole USC community. Another
member of the university community such as a friend, classmate, advisor, or
faculty member can help initiate the report, or can initiate the report on behalf of
another person. The Center for Women and Men http://www.usc.edu/studentaffairs/
cwm/ provides 24/7 confidential support, and the sexual assault resource
center webpage http://sarc.usc.edu describes reporting options and other

resources.
18. Support Systems
A number of USCs schools provide support for students who need help with
scholarly writing. Check with your advisor or program staff to find out
more. Students whose primary language is not English should check with the
American Language Institute http://dornsife.usc.edu/ali, which sponsors courses
and workshops specifically for international graduate students. The Office of
Disability Services and Programs
http://sait.usc.edu/academicsupport/centerprograms/dsp/home_index.html
provides certification for students with disabilities and helps arrange the relevant
accommodations. If an officially declared emergency makes travel to campus
infeasible, USC Emergency Information http://emergency.usc.edu will provide
safety and other updates, including ways in which instruction will be continued by
means of blackboard, teleconferencing, and other technology.