MUSIC Algorithm For Young Mathematicians and Engineers
MUSIC Algorithm For Young Mathematicians and Engineers
Engineers
Raveen De Silvaa, Amir Antonira, Aleksandar Ignjatovica, and Chamith Wijenayakeb
School of Computer Science, University of New South Wales, Sydney, Australiaa
School of Information Technology and Electrical Engineering, University of Queensland, Brisbane, Australiab
Corresponding Author Email: [email protected]
ABSTRACT
CONTEXT
Training of young mathematicians most often focuses only on mathematical and computational
methods, without giving students a deeper insight into foundations and context of engineering,
which is the most common source of the problems they will be tackling. On the other hand,
engineering students are often taught how to manipulate mathematical formulas without having a
deeper understanding of the underlying first principles. We believe that there are two reasons for
such an unhappy state of modern education. (1) In the modern world, we have become super-
specialised, and for one individual to frequently cross the boundaries between fields is not
generally perceived as a good career move. Education is often reduced to professional training,
where the aim is not to nourish deeper understanding, but just to acquire sufficient skills to perform
certain tasks. (2) Specialist textbooks are often replete with discipline specific technical jargon.
PURPOSE OR GOAL
While one might see this situation as the most cost-effective way to provide industry with workforce
in the short term, we believe that this approach will eventually erode our ability to make paradigm
shifts. Perhaps it is time to take a bit of the attitude of the Renaissance and of Enlightenment and
give our education just a broader, more humanistic focus than just providing a skill set.
APPROACH OR METHODOLOGY/METHODS
As an example for how this can be accomplished, we offer our presentation of the MUSIC (Multiple
Signal Classification) frequency estimation algorithm, free of any signal processing jargon, not
requiring absolutely any knowledge of signal processing but only knowledge of basic linear
algebra. We believe that such an approach can give a small hint how to bridge the chasm between
education of mathematicians and education of signal processing engineers.
CONCLUSIONS/RECOMMENDATIONS/SUMMARY
Engineering education should be more integrated, unifying teaching a good deal of science with
teaching practical engineering. We must resist the pressures to be “the industry of higher
education” and be what we used to be – academia, educating not only practitioners of
sophisticated skills but also independent and creative thinkers and innovators.
KEYWORDS
Science in engineering, wholistic education, thinkers, innovators
Introduction
Education for Mathematics and Engineering students often have different focus, thus creating two
different worlds for mathematicians and engineers. Students in Engineering often taught to use and
manipulate various mathematical formulas with omission of proofs and first principles. Students in
mathematics often learn underlying first principles, proofs, mathematical and computational
methods with limited exposure in the context of engineering problems. Engineering textbooks, as a
rule, try to avoid excursions into mathematical theory as much as possible, while mathematical
textbooks usually state the relevant equations without placing them into a proper engineering
context. In fact, we feel that engineers and mathematicians mostly live in two separate worlds,
often unwilling to cross the boundaries between fields or at least trying to minimise such
excursions. Examples are abundant: young mathematicians most often learn harmonic analysis
without even a mention of one of the most spectacular applications of harmonic analysis, namely
signal processing. On the other hand, young electrical engineers are most often taught the
Discrete Fourier Transform (DFT) by simply giving them the relevant formulas, without ever telling
them that the DFT amounts to a change of basis of the underlying signal space.
As an approach to reduce this disconnect, we explore the possibility of redesigning learning
resources (such as self-paced asynchronous tutorials), which take a commonly used textbook
application from Engineering (e.g., a signal processing algorithm from Electrical Engineering) and
explain the underlying theory and operation, purely based on mathematical details (such as linear
algebra) without any prior knowledge in signal processing. As an example, we present the MUSIC
(Multiple Signal Classification) and the root-MUSIC frequency estimation algorithms used in
standard signal processing, essentially without any reference to traditional signal processing
concepts and without using any signal processing terminology, thus making these algorithms
accessible to all students who have studied basic linear algebra, regardless of their field of study.
Students of signal processing can also benefit from reading this tutorial which presents very clearly
the underlying mathematical foundations of these algorithms.
While the rest of this paper takes the form a mathematical tutorial, the main objective of this paper
is to demonstrate how sophisticated technical algorithms in Engineering can be presented in a
manner making them accessible to a wide audience, bypassing technical jargon and often
requiring minimal background preparation.
Background
In the modern university, students often specialise very early in their studies, and have little
substantial engagement with other fields. Henderson and Broadbridge (Henderson & Broadbridge,
2009) describe the state of engineering mathematics education in Australia, including the
challenges presented by the diversity of mathematical experience and proficiency in the student
body, the need to incorporate computing and statistics, and administrative issues such as class
sizes, assessment and budgets. The content and skills taught in these courses are fundamental to
the quantitative methods used in various engineering disciplines, as discussed by Maass et al.
(Maass, Geiger, Romero Ariza, & Goos, 2019) who write that “STEM education in general, and
mathematics education in particular, can contribute to preparing individuals better for twenty-first
century challenges”. However, it is well documented that many engineering students struggle in
these mathematics courses, and they often perceive the content as unnecessarily abstract and of
little relevance to their engineering studies. This issue has received much attention in the literature,
such as case studies of the problem-based learning approach by Flegg et al. (Flegg, Mallet, &
Lupton, 2012) and Bischof et al. (Bischof, Bratschitsch, Casey, & Rubesa, 2007). In addition,
authors such as Ooi (Ooi, 2007) and Klingbeil et al. (Klingbeil, Mercer, Rattan, Raymer, &
Reynolds, 2004) write on the matters of what mathematics should be presented to engineering
students and at what stage of their studies, with the latter advocating for the traditional calculus
sequence to be delayed in favour of teaching freshmen “only the math they really needed to know
in order to progress into their sophomore and junior years”.
Proceedings of AAEE 2022 Western Sydney University, Sydney, Australia, Copyright © Raveen De Silva, Amir Antonir, Aleksandar
Ignjatovic, Chamith Wijenayake, 2022.
In this paper, we primarily examine the converse problem: that of demonstrating engineering
applications to mathematics students who may be otherwise more interested in mathematics for its
own sake. This is much less widely studied, but we believe it to be important in order to both
diversify the interests and expertise of mathematics students and foster effective interdisciplinary
collaboration. Many mathematics students are not aware of the power of the material they study in
solving real-world problems, and here, we demonstrate how they might be introduced to the
application of linear algebra in signal processing, without any of the jargon from signal processing,
that might be off-putting to a student without any formal training in electrical engineering.
The rest of the paper presents our example tutorial of MUSIC algorithm based on linear algebra.
Here, 𝑗 is the imaginary unit and real numbers 𝐴! , 𝜔! and 𝜑! are the amplitude, the frequency,
and the phase of the 𝑘 "# component, respectively; we assume that 𝐴! > 0 and −𝜋 < 𝜔! < 𝜋,
0 < 𝜑! < 2𝜋 for all 1 ≤ 𝑘 ≤ 𝑛.
We also assume that there exists a sequence of samples 𝑠(𝑡 + 𝑚) of such a signal, taken at
consecutive instants a unit distance apart, starting with an instant t. Let us now form a linear
combination of n + 1 such consecutive samples with coefficients 𝑐, , … , 𝑐) to be specified below.
Using (1)Error! Reference source not found., after some simplification we obtain
) ) ) ) )
-
' 𝑐- 𝑠(𝑡 + 𝑚) = ' 𝑐- ' 𝐴! 𝑒 "($! (%&-)&'! ) = ' 2 ' 𝑐- 3𝑒 "$! 4 5 𝐴! 𝑒 "($! %&'! )
-*, -*, !*+ !*+ -*,
(2)
Consider now a polynomial 𝑃(𝑧) with the leading coefficient 1, given by the product 𝑃(𝑧) =
3𝑧 − 𝑒 "$" 4 … 3𝑧 − 𝑒 "$# 4. Let 𝑐, , … , 𝑐) be the coefficients of this polynomial, such that
) )
-
' 𝑐- 𝑧 = 93𝑧 − 𝑒 "$! 4.
-*, !*+
Then, since the right-hand side of (2) is of the form ∑)!*+ 𝑃3𝑒 "$! 4 𝐴! 𝑒 "($! %&'! )
and since 𝑒 "$" , … , 𝑒 "$# are the roots of 𝑃(𝑧), the right-hand side (2) of will be equal to zero for all
real t, and vice versa: since complex exponentials with distinct frequencies are linearly
independent functions, if the right side of (2) is equal to zero for all real t, then 𝑒 "$! must be the
roots of the polynomial 𝑃(𝑧). Consequently, in order to find the unknown frequencies 𝜔! it is
enough to find then coefficients 𝑐, , … , 𝑐) such that for all 𝑡,
)
' 𝑐- 𝑠(𝑡 + 𝑚) = 0
-*,
(3)
Proceedings of AAEE 2022 Western Sydney University, Sydney, Australia, Copyright © Raveen De Silva, Amir Antonir, Aleksandar
Ignjatovic, Chamith Wijenayake, 2022.
and then find the roots of the associated algebraic equation ∑)-*, 𝑐- 𝑧 - = 0; such roots lie on the
unit circle and their arguments are the frequencies sought.
To find a non-zero vector 𝒄 = 𝑐, , … 𝑐) such that
(3) holds for all t, we instantiate
(3) from 𝑡 = 1 to = 𝑛, and if 2𝑛 samples 𝑠(1), 𝑠(2), … 𝑠(2𝑛) of the signal 𝑠(𝑡) are available, we
obtain a system of linear equations in unknown coefficients 𝑐, , … 𝑐) of the form
𝑠(1) 𝑠(2) ⋯ 𝑠(𝑛 + 1) 𝑐,
⎛ 𝑠(2) 𝑠(3) ⋯ 𝑠(𝑛 + 2) ⎞ ⎛ 𝑐+ ⎞
⋱
⎜ ⋮ ⋮ ⋮ ⎟⎜ ⋮ ⎟ = 𝟎
𝑠(𝑛 − 1) 𝑠(𝑛) ⋯ 𝑠(2𝑛 − 1) 𝑐).+
⎝ 𝑠(𝑛) 𝑠(𝑛 + 1) ⋯ 𝑠(2𝑛) ⎠ ⎝ 𝑐) ⎠
(4)
which, in absence of any noise, we can solve exactly. This is the Prony method, dating back to
year 1795 [Error! Reference source not found.], which is summarised in Fig. 1.
Proceedings of AAEE 2022 Western Sydney University, Sydney, Australia, Copyright © Raveen De Silva, Amir Antonir, Aleksandar
Ignjatovic, Chamith Wijenayake, 2022.
Assume now that we have 𝑁 > 2𝑛 equidistant samples 𝑓(1), . . . , 𝑓(𝑁) of a signal 𝑓(𝑡) = 𝑠(𝑡) +
3
𝜈(𝑡) which is a sum of a signal 𝑠(𝑡) as in (1) and noise 𝜈(𝑡). We can still form a Hankel matrix 𝐌/
obtained from matrix 𝐌/0 by replacing samples of 𝑠(𝑡) by the corresponding noisy samples of 𝑓(𝑡).
3
Due to the presence of noise, 𝐌/ will generally have a full rank of 𝑛 + 1, and thus all singular
3
values of 𝐌/ will be non-zero. We now take the rightmost singular vector, which corresponds to
the smallest singular value, as an approximation of the rightmost singular vector if no noise were
present, and its components 𝐯)&+ (𝑚 + 1) as approximations of the values of 𝑐- for which
equation (3) holds.
3
As is well known, the right singular vectors of matrix 𝐌/ are the eigenvectors of the product matrix
3 3 ∗ 3
𝐀/ = 3𝐌/ 4 𝐌/ , where 𝐌∗ denotes the conjugate transpose of 𝐌. Note that, in our particular case,
3 3
(1/𝑁)𝐀/ is just the auto-covariance matrix of the samples of the noisy signal 𝑓(𝑡). Since matrix 𝐀/
3
is of size (𝑛 + 1) × (𝑛 + 1) and since 𝑁 is usually much larger than 𝑛, 𝐀/ is of much smaller size
3 3
than 𝐌/ . Consequently, finding the eigen decomposition of 𝐀/ is a computationally lighter task
3
than finding the singular value decomposition of 𝐌/ . Note that this benefit, however, is often offset
3 3 ∗ 3
by the cost of the computation of the matrix product 𝐀/ = 3𝐌/ 4 𝐌/ . In this way we obtain the
Pisarenko frequency estimation algorithm, which can be summarised as follows and in Fig. 2:
3 3 ∗ 3 3
Compute the matrix 𝐀/ = 3𝐌/ 4 𝐌/ and obtain its eigenvalue decomposition 𝐀/ = 𝐐𝚲𝐐∗ . Take the
3
rightmost eigenvector 𝐪)&+ which corresponds to the smallest eigenvalue of 𝐀/ and solve the
associated algebraic equation ∑)-*, 𝐪)&+ (𝑚 + 1)𝑧 - = 0; the arguments of its 𝑛 roots are taken
as estimates of the unknown frequencies 𝜔! of the complex exponentials which are the 𝑛
components of the signal 𝑠(𝑡).
Proceedings of AAEE 2022 Western Sydney University, Sydney, Australia, Copyright © Raveen De Silva, Amir Antonir, Aleksandar
Ignjatovic, Chamith Wijenayake, 2022.
0 0
values of 𝐌- , we would have 𝐌- 𝐯4 = UΣ𝐕 ∗ 𝐯𝒊 = 0. Thus, each of the right singular vectors 𝑣4 , 𝑛 <
𝑖 ≤ 𝐾, produces an equation of the form
6.+
' 𝐯4 (𝑚 + 1)𝑓(𝑡 + 𝑚) = 0
-*,
that holds for samples 𝑓(𝑡), . . . , 𝑓(𝑡 + 𝐾 − 1) for all integers 1 ≤ 𝑡 ≤ 𝑁 − 𝐾 + 1. Consequently, in the
absence of any noise, for every 𝑛 < 𝑖 ≤ 𝐾 and all 1 ≤ 𝑡 ≤ 𝑁 − 𝐾 + 1, equation (2) would imply
6.+ ) 6.+
"'!
' 𝐯4 (𝑚 + 1)𝑓(𝑡 + 𝑚) = ' 𝐴! 𝑒 2 ' 𝐯4 (𝑚 + 1)(𝑒 "$! )- 5 𝑒 "$$ % = 0.
-*, !*+ -*,
(7)
%
If 𝑁 − 𝐾 + 1 ≥ 𝑛, since the Vandermonde matrix 𝐕 = f3𝑒 "$! 4 ∶ 1 ≤ 𝑘, 𝑡 ≤ 𝑛i is always
nonsingular if all 𝜔! are distinct, equation (7) would imply that 𝑒 "$! must be among the 𝐾 − 1
many roots of each of the polynomials 𝑃4 (𝑧), where
6.+
Proceedings of AAEE 2022 Western Sydney University, Sydney, Australia, Copyright © Raveen De Silva, Amir Antonir, Aleksandar
Ignjatovic, Chamith Wijenayake, 2022.
(10)
To avoid such a numerical search, the root-MUSIC algorithm instead uses the fact that complex
numbers 𝑧 which are close to the unit circle satisfy 𝑧̅ ≈ 𝑧 .+ and thus instead explicitly solves the
following equation, conveniently reducible to an algebraic (i.e., polynomial) equation
6
u8 (𝑧 .+ ) = 0
' 𝑃4 (𝑧)𝑃
4*)&+
(11)
! 𝑖 (𝑧) denotes the polynomial obtained from the polynomial 𝑃𝑖 (𝑧) by taking the complex
where 𝑃
conjugates of the coefficients of 𝑃𝑖 (𝑧).
It is clear that if 𝑧4 is a root of equation (11) then so is 𝑧lllll
.+ ; thus, the roots of (11) come in pairs of
.+
the form: v𝑝4 𝑒 "$ , 𝜌4 𝑒 "$ y. The root-MUSIC algorithm picks 𝑛 pairs of such roots that lie closest to
the unit circle; the arguments of these pairs are taken as the estimates of the frequencies 𝜔! (1 ≤
𝑘 ≤ 𝑛) of the 𝑛 components of 𝑠(𝑡), as illustrated in Fig.4(b)).
The interested reader can find more details in (Stoica & Moses, 1979) (Pisarenko, 1973).
(a) (b)
Required roots of
equation (11)
Figure 4: (a) Example peak profile of function 𝑹(𝝎) given in equation (10); (b) illustration of roots of
equation (11) as pairs of complex numbers lying close to unit circle.
Discussion
The tutorial explanations given above were presented to a group of students from which 7 survey
responses were recorded. Although the response rate to the feedback survey is quite low, 70% of
Proceedings of AAEE 2022 Western Sydney University, Sydney, Australia, Copyright © Raveen De Silva, Amir Antonir, Aleksandar
Ignjatovic, Chamith Wijenayake, 2022.
the responses agreed that “the tutorial provides a clear description about the MUSIC algorithm,”
and 56% agreed that “I obtained a good understanding about the MUSIC algorithm after going
through the tutorial.” Survey participants also thought that providing more visual interpretation of
some of the linear algebra operations will be helpful from a student perspective.
Where traditional methods to teach signal processing algorithms are grounded in terminology from
electrical engineering and aim to solve concrete problems, ours is deliberately abstract. We seek to
first demonstrate to mathematics students how the singular value decomposition can be used to
identify the dominant frequencies in a noisy signal numerically, to be later supplemented by
discussion of the application of these methods in digital signal processing. We believe that this
approach is different to traditional classroom teaching of such signal processing algorithms.
We envision that this material could be presented in courses on linear algebra, which often
introduce the singular value decomposition, but may not contextualise it by demonstrating its
application to practical engineering problems. By first relating the theory to a challenging
mathematical problem, we aim to inspire mathematics students to consider how their work is
related to that undertaken in other disciplines.
This tutorial could also be applied to courses on mathematical computing, as the implementation
and analysis of these algorithms in a software package such as MATLAB is an instructive exercise
and provides a tangible outcome from linear algebra subroutines.
References
Bischof, G., Bratschitsch, E., Casey, A., & Rubesa, D. (2007). Facilitating Engineering Mathematics
Education by Multidisciplinary Projects. Paper presented at the ASEE Annual Conference & Exposition,
Honolulu, Hawaii.
Flegg, J., Mallet, D., & Lupton, M. (2012). Students' perceptions of the relevance of mathematics in
engineering. International Journal of Mathematical Education in Science and Technology, 717-732.
Henderson, S. & Broadbridge, P. (2009). Engineering Mathematics Education in Australia. MSOR
Connections, 9(1), 12-17.
Klingbeil, N., Mercer, R., Rattan, K., Raymer, M., & Reynolds, D. (2004). Rethinking Engineering
Mathematics Education: A Model for Increased Retention, Motivation and Success in Engineering. Paper
presented at the ASEE Annual Conference & Exposition, Salt Lake City, Utah.
Maass, K., Geiger, V., Romero Ariza, M., & Goos, M. (2019). The Role of Mathematics in Interdisciplinary
STEM education. ZDM, 51, 869-884.
Ooi, A. (2007). An analysis of the Teaching of Mathematics in Undergraduate Engineering Courses. Paper
presented at the 18th Conference of the Australasian Association for Engineering Education, Melbourne,
Victoria.
Pisarenko, V. F. (1973). The retrieval of harmonics from a covariance function. Geophysical Journal
International, 33, 347-366.
Prony, R. (1795). Essai experimentale et analytique. J. Ecole Polytechnique, 2, 24-76.
Proceedings of AAEE 2022 Western Sydney University, Sydney, Australia, Copyright © Raveen De Silva, Amir Antonir, Aleksandar
Ignjatovic, Chamith Wijenayake, 2022.
Stoica, P. & Moses, R. (1979). Introduction to Spectral Analysis (1st ed.). Upper Saddle River, NJ: Prentice
Hall.
Copyright statement
Copyright © 2022 Raveen De Silva, Amir Antonir, Aleksandar Ignjatovic, Chamith Wijenayake: The authors assign to the Australasian
Association for Engineering Education (AAEE) and educational non-profit institutions a non-exclusive licence to use this document for
personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors
also grant a non-exclusive licence to AAEE to publish this document in full on the World Wide Web (prime sites and mirrors), on
Memory Sticks, and in printed form within the AAEE 2022 proceedings. Any other usage is prohibited without the express permission of
the authors.
Proceedings of AAEE 2022 Western Sydney University, Sydney, Australia, Copyright © Raveen De Silva, Amir Antonir, Aleksandar
Ignjatovic, Chamith Wijenayake, 2022.