ECE 6250: Advanced Topics in Digital Signal Processing Fall 2011 Syllabus Updated: August 22, 2011

Summary
ECE 6250 is a general purpose, advanced DSP course designed to follow an introductory DSP course. The central theme of the course is the application of tools from linear algebra to problems in signal processing.

Prerequisites
A senior-level introductory course in digital signal processing such as ECE 4270. Students should be familiar with the fundamentals of linear algebra and have had exposure to basic probability and statistics. Students should also have basic MATLAB programming skills.

Instructor
Justin Romberg Email: [email protected] Oce: Centergy 5245 Oce phone: 404-894-3930 Oce hours: I will typically hold scheduled oce hours the day before homework is due. More details will be provided soon. I am also available to meet in Centergy 5245 by appointment.

Teaching Assistant
A TA has not been assigned for this course yet. Hopefully this will be taken care of by the end of the rst week of class.

Grading
20% Homework, 11 (1) assignments + 1 solution, see below 25% Quiz #1 (late September) 25% Quiz #2 (early November) 25% Final exam 5% Attendance, see below

Homework
Homework will be assigned weekly (approximately). Homework will be turned in at the beginning of lecture. Late homework will get zero credit. Students are encouraged to discuss homework problems with one another, however each student must write up and turn in their own solutions. Unauthorized use of any previous semester course materials, such as tests, quizzes, homework, projects, and any other coursework, is prohibited in this course. Using these materials will be considered a direct violation of academic policy and will be dealt with according to the GT Academic Honor Code. The homework assignments will be hard; many of them will require signicant amounts of time and eort to complete. But this is really where most of the learning takes place. You will get out of the assignments what you put into them. Students who complete all of the assignments in full will be rewarded with a deep understanding of the role that linear algebra plays in modern signal processing (among other things). Eectively, homework is worth much more than 20% of your grade. In teaching this class, I have yet to see a case where a student does not put eort into the homework assignments but does well on the exams.

Lecture
Lectures are Monday and Wednesday from 3:05-4:25p in Klaus 2443. Lecture attendance is mandatory and will count towards your grade. A sign in sheet (starting in week 2) will be passed around at every lecture; please sign next to your name and your name only.

Web page and T-square

Course information, notes for the lectures, homework assignments, and supplemental materials will be posted here. We will also use T-square. Here students can download videos of the lectures, participate in discussion forums about the homework, and review their grades.

Distance learning
Students enrolled in the video section (ECE6250Q) will have a one-week lag for turning in assignments and tests. I will also be available to answer questions via email or the telephone. For basic procedures and policies, please visit the DLPE website at http://www.dlpe.gatech.edu/ .

Text
There is no required text. A good reference for the course material is Moon and Stirling: Mathematical Methods and Algorithms for Signal Processing http://tinyurl.com/kvelng Below is a list of books that I have found helpful over the years for learning (and teaching) the material in this class. Strang: Linear Algebra and its Applications http://tinyurl.com/my5ufk Strang: Introduction to Applied Mathematics http://tinyurl.com/nj6r5d Horn and Johnson: Matrix Analysis http://tinyurl.com/kvtxkp Laub: Matrix Analysis for Scientists and Engineers http://tinyurl.com/3ev7crf Young: An Introduction to Hilbert Space http://tinyurl.com/kt9jzu Nguyen and Strang: Wavelets and Filter Banks http://tinyurl.com/lsjyox Scharf: Statistical Signal Processing http://tinyurl.com/3c9qc6p

Outline
1. Sampling and lter banks (a) The Shannon-Nyquist sampling theorem (b) Multirate digital signal processing (c) Filter banks and the discrete wavelet transform 2. Signal representations in vector spaces (a) Linear vector spaces, linear independence, and basis expansions (b) Norms and inner products (c) Orthobases, the reproducing formula, and Parsevals theorem (d) Signal approximation in an inner product space (e) Gram-Schmidt and the QR decomposition 3. Linear inverse problems (a) Introduction to linear inverse problems, examples (b) The singular value decomposition (SVD) (c) Least-squares solutions to inverse problems and the pseudo-inverse (d) Stable inversion and regularization (e) Weighted least-squares and linear estimation (f) Least-squares with linear constraints 4. Computing the solutions to large-scale least-squares problems (a) Structured matrices (b) Steepest descent (c) The conjugate gradient method 5. Low-rank updates for streaming solutions to least-squares problems (a) Recursive least-squares (b) The Kalman lter 6. Matrix approximation using least-squares (a) Low-rank approximation of matrices using the SVD (b) Total least-squares (c) Principal components analysis 7. Beyond least-squares (topics as time permits) (a) Norm approximation problems (b) Non-smooth regularization for linear inverse problems (c) Independent components analysis (d) Linear programming for
1

and

problems