Proceedings ICMC|SMC|2014 14-20 September 2014, Athens, Greece

New Tools for Aspect-Oriented Programming in Music and Media

Programming Environments

John MacCallum, Adrian Freed, David Wessel

Center for New Music and Audio Technologies
Department of Music
University of California, Berkeley
{john,adrian,wessel}@cnmat.berkeley.edu

ABSTRACT Of these principles, 2, 3, 6, and 7 are perhaps the most im-

portant for our purposes. Since the creation of a work of
Media/arts programming is often experimental and exploratory art is a speculative process, a clear and well-defined spec-
in nature and requires a flexible development environment ification for a piece of software is rarely possible. Itera-
to enable continually changing requirements and to facili- tive development where the software is tested and evalu-
tate iterative design in which the development of software ated in the context of the piece being created is essential
impacts the design of a work of art which in turn pro- to allow the software and the concept of the work of art to
duces new requirements for the software. We discuss agile co-evolve. To promote this type of exchange, the develop-
development as it relates to media/arts programming and ment environment must be one that is flexible, not brittle,
present aspect-oriented programming and its implementa- and one that welcomes potentially drastic change as the
tion in Max/MSP using Open Sound Control and the odot result of incremental use and evaluation. Aspect-oriented
library as tool for mobilizing the benefits of agile develop- programming can limber up the development environment
ment. when used sparingly and judiciously.

1.1 Aspect-Oriented Programming

1. INTRODUCTION
Before providing a description of aspect-oriented program-
Media/arts programming is often speculative in nature and ming, it is useful to define some terms that are used through-
its practice is most closely related to that of agile devel- out the literature [4].
opment [1] in the software engineering community. The
following principles constitute (with some slight modifica- 1.1.1 Terminology
tions for media/arts development) agile programming [2, The following terms are from the AOP literature
3]:
1. The person for whom the development is being done Cross-cutting concerns Aspects of a program that cut across
(often oneself) should be satisfied through early and or are interwoven among many different parts of a
continuous delivery of valuable software. program.
2. Welcome changing requirements, even late in de- Advice Additional behavior applied to data in the context
velopment. Agile processes embrace change as the of an aspect.
artist adapts his/her vision of the project based on
iterations of the software. Join point A point in the control flow of a program (in
3. Deliver working software frequently. dataflow languages like Max/MSP, PD, or Ptolemy
4. Artists and developers must work together often through- II [5] 1 these points are inlets and outlets of data flow
out the project. actors).
5. Working systems are the primary measure of progress.
6. Agile processes promote sustainable development. Pointcut A set of join points that may have advice associ-
The artists and developers should be able to main- ated with them.
tain a constant pace indefinitely. 1.1.2 Description
7. Continuous attention to technical excellence and good
design enhances agility. “Aspect-oriented programming (AOP) is a programming
8. Simplicity—the art of maximizing the amount of work methodology [which separates out] cross-cutting concerns
done—is essential. [. . . ] from the main code of the actions to which the con-
cerns apply.”[6] Some examples of crosscutting concerns
Copyright: 2014
c John MacCallum et al. This is
that are useful in the context of real-time media/arts pro-
an open-access article distributed under the terms of the
gramming are:
Creative Commons Attribution 3.0 Unported License, which permits unre-
- Logging
stricted use, distribution, and reproduction in any medium, provided the original
- Visualization
author and source are credited. 1 http://ptolemy.eecs.berkeley.edu

- 1587 -
 Proceedings ICMC|SMC|2014 14-20 September 2014, Athens, Greece

- Structural analysis things: a) modification to the existing patch which carries

- Commentary with it the loss of any stored internal state and the potential
- Scaffolding of introducing new bugs or behavior that can impede the
- Stream capture search for the bug, or b) foresight during the initial imple-
- Performance profiling mentation of the patch resulting in debugging components
- Debugging, printing and tracing that will aid in our current situation.The former is cumber-
- Input/Output validation and assertion some, and the latter, when generalized to all types of de-
- Dynamic code injection without modifying existing bugging situations, requires the kind of forward thinking
(possibly running) code design that is particularly difficult in dynamic and specula-
Most modules in a complex program will need some of tive work.
the features itemized above at some point in the develop- One of the tenets of AOP is that the programmer should
ment process. AOP enables those elements of the program be oblivious to future aspects that may be applied at a later
to be injected when necessary and easily removed when date, obviating the need for the programmer in our exam-
they are no longer needed without modifying the code to ple to predict future debugging situations. One must still
which they are applied. AOP frees programmers from the ensure that each inlet and outlet of the abstraction is con-
need to foresee functionality and, often obviates the need nected to an o.in or o.out, but assuming those hooks are in
for programmers to revisit code to extend or modify its be- place, the input to a patch may be produced as follows (see
havior. By providing each Max abstraction with the proper figure 2).
hooks, we can quickly and unobtrusively add additional 1. Create a new patch and instantiate o.aspect.receive.
behavior (“advice”) at various points in the program (“join 2. Filter the stream of bundles based on the port type
points”) without modifying existing code. Further, AOP (and possibly other contextual information).
obviates the need to remember a number of ad hoc systems 3. Display the data in an appropriate way if it matches
for mundane functionality such as printing to the Max win- the contextual criteria.
dow. 4. When the visualization is no longer necessary, save
the aspect in case it may be useful in the future, and
simply close the window. If it becomes useful again,
2. SIMPLE EXAMPLE
reopen it.

Body Advice
Body

(a) Max abstraction (b) Max abstraction with hooks

Figure 1: On the left, we see the typical dataflow through Figure 2: o.aspect.print
an abstraction in Max. On the right, the use of o.in and
o.out to forward incoming and outgoing data to aspects.
4. MAX/MSP IMPLEMENTATION
In figure 1a, we see the skeleton of a module in Max, The implementation of AOP in Max that we introduce here
while in figure 1b o.in and o.out provide join points where makes use of Open Sound Control [7] as a rich, composite
advice can be applied. Data enters a module and is for- data type, and the odot library [8] for providing contex-
warded by o.in to o.aspect.receive where advice is applied. tual information and high-level processing of OSC data.
After processing, it is sent back to where it came from, in AOP is implemented in Max using a pair of Max “abstrac-
this case o.in, by o.aspect.receive. o.in then forwards the tions” or shims called o.in and o.out which are placed im-
(possibly modified) data to the body of the Max abstrac- mediately after and before each inlet and outlet in a mod-
tion. o.out behaves identically to o.in with the exception of ule, respectively. o.in and o.out are thin wrappers around
its contextual information. o.port which collects contextual information from its en-
vironment such as the name of the module it is in and the
name of the parent patch, as well as the arguments to those
3. APPLICATIONS
patches. This data is added to the OSC bundle along with a
As programs grow in complexity, the need to understand “return address” and sent, using Max’s built-in “send” ob-
what is going on inside of submodules nested deep in the ject, to the global location “o.aspect”, i.e., [send o.aspect].
patcher hierarchy can present serious difficulties. For ex- We provide two additional abstractions that aid in writ-
ample, we may suspect that something is going wrong in ing aspects: o.aspect.receive which simply wraps [receive
an abstraction and wish to see the values that are being sent o.aspect], and o.aspect.send which uses the “return address”
into it as a way to determine whether the problem occurs to set the forward object to send to that location (see figure
inside or outside the abstraction. This requires one of two 3).

- 1588 -
 Proceedings ICMC|SMC|2014 14-20 September 2014, Athens, Greece

ule.
The following enumerated items correspond to those in
figure 4.
1. OSC bundles are processed and all other non-OSC
data is simply passed through untouched.
2. The port type, “inlet” or “outlet”, is bound to /o port/type
and blended into the incoming OSC stream.
3. The context information for this port is blended into
the incoming OSC stream.
4. The bundle is passed to the join point.
Figure 3: o.aspect.send 5. All contextual information is stripped off before out-
putting.

To create an aspect, one builds a Max/MSP patch and 4.2 o.aspect.joinpoint

instantiates o.aspect.receive and o.aspect.send (both with
no arguments). o.aspect.receive will produce all messages
sent to the location “o.aspect”. The bundle can then be
processed according to the advice that this particular aspect 1

provides. Certain aspects may choose to inject data into

the bundle (e.g., profiling data) in which case the aspect 2
should send the modified bundle back using o.aspect.send.
If no modifications are made, this step may be skipped and
a copy of the original bundle will be produced after the join
3
point.
The following sections describe the implementation of
o.port and o.aspect.joinpoint.

4.1 o.port
4

Figure 5: o.aspect.joinpoint

3 o.aspect.joinpoint (see figure 5) is responsible for dis-

patching incoming OSC bundles to any pointcuts that may
be instantiated. If no pointcuts are in place, a copy of the
4 bundle is simply passed through unchanged.
The following enumerated items correspond to those in
5 figure 5.
1. OSC bundles are processed and all other non-OSC
data is simply passed through untouched.
Figure 4: o.aspect.port 2. A copy of the incoming bundle is stored in o.collect.
This will be sent out as is if no data is returned,
o.port (see figure 4) is responsible for gathering contex- or a union operation will be performed producing
tual information about its environment, blending it into the a bundle containing the original data and any data
bundle, passing it to o.aspect.joinpoint, and outputting the added by any aspects. After the bundle is passed to
result. o.port is a general mechanism that makes use of o.collect, it is processed by the following steps.
o.aspect.joinpoint and also serves as the location for imple- 3. A “return address” is created using the unique nu-
menting other hooks to extend the functionality of a mod- merical id created using the “0” lexical substitution

- 1589 -
 Proceedings ICMC|SMC|2014 14-20 September 2014, Athens, Greece

variable and blended into the OSC bundle. 4. Choose a random frequency from the spectrum us-
4. Contextual information is generated using o.context ing the amplitudes as a (categorical) probability dis-
and blended into the OSC bundle. tribution.
5. The bundle is sent to the named global location “o.aspect”. 5. Sonify a grain at the chosen frequency.
6. If a bundle is received here, the “return address” is
removed and it is sent to o.collect which will com-
bine it with the original. 1

3
5. A MORE DETAILED EXAMPLE

Figure 7: Choose a frequency at random using a list of

4
amplitudes as probabilities.

The module called spec-sample draws a random sample

from the spectrum and is implemented as follows (see fig-
5 ure 7).
1. Inlet with o.in.
2. Store a copy of the bundle in order to blend derived
data into it.
3. Extract the list of amplitudes.
Figure 6: A granular synthesizer that chooses the fre- 4. Treat the list of amplitudes as a probability mass
quency of each grain from a spectrum produced by function, convert it to a cumulative distribution func-
sigmund∼ using the amplitudes of each frequency com- tion, and draw a random sample from it.
ponent as a probability distribution. 5. Assign the frequency corresponding to the random
index to the address /freq.
6. Output the bundle, passing through o.out first.
In this section, we use the implementation of a granular
synthesizer to discuss the use of aspects in the develop- 5.1 Visualization
ment process. The granular synthesizer, seen in figure 6,
consists of two parts: code that chooses the frequency of a While developing this granular synthesizer, we may want
grain, and the code to sonify the grain. In this example, we to visualize its output. Normally, we would patch some-
take the spectral output of Miller Puckette’s sigmund∼ ex- thing together in the main patch, however, such ad hoc
ternal 2 and choose frequencies at random from it using the work is often discarded when not needed in order to clean
amplitudes as probabilities. This ensures that more grains the patch up and optimize it for efficiency. AOP can assist
will be set to those components that had a greater ampli- here as seen in figure 8. We instantiate o.aspect.receive
tude in the spectrum. The sonification can be done with and filter any incoming OSC data by looking specifically
any suitable polyphonic synthesizer—in our case, we use a for bundles that come from outlets and are sent from mod-
simple enveloped sine wave. The following is a description ules called “spec-vis”. We then interleave the frequencies
of each component of figure 6. and amplitudes for display with resdisplay 3 and display
1. Encapsulated logic for playback of sound files with the data bound to /freq using multislider.
groove∼.
2. Capture output from sigmund∼ and encode as OSC. 5.2 Extension and Experimentation
3. Drive the granular synthesizer with a clock indepen- We may wish to add additional behavior to our program,
dent of the rate of output of sigmund∼. for example, spectral smearing which we could implement
2 http://crca-archive.ucsd.edu/ tapel/software.html 3 http://cnmat.berkeley.edu/downloads

- 1590 -
 Proceedings ICMC|SMC|2014 14-20 September 2014, Athens, Greece

Aspect oriented programming is a relatively recent paradigm

with a growing community of users exploring where it can
be effective. Our contribution in this work is to bring the
paradigm to the music and intermedia communities. As
well as having noticeable positive effects on our own pro-
ductivity we have discovered that this style of program-
ming requires extensions to the core programming environ-
ments especially in the area of introspection. We have also
found interesting opportunities to extend aspect-oriented
programming afforded by visual programming environments.
For example, it is effective to write aspects that change the
colors of Max/MSP object boxes,text and patch chords to
contextualize state changes and program flow.

Acknowledgments
This work was supported in part by the TerraSwarm Re-
Figure 8: An aspect used to visualize the data computed
search Center, one of six centers supported by the STAR-
in spec-sample.
net phase of the Focus Center Research Program (FCRP) a
Semiconductor Research Corporation program spon-sored
by adding a random value to the chosen frequency. We by MARCO and DARPA.
may want to experiment with different families of proba-
bility distributions, and ultimately we may wish to discard 7. REFERENCES
this behavior if it proves to be uninteresting. Rather than
[1] A. Cockburn, Agile Software Development: The Coop-
perform many edits on a working patch, we can contain
erative Game, 2nd ed. Addison-Wesley Professional,
this speculative work in an aspect as seen in figure 9.
2006.
We first look for OSC bundles that came from the outlet
of the spec-sample patch. We then blend in the name of a [2] R. E. Filman, T. Elrad, S. Clarke, and M. Akşit, Eds.,
probability distribution and its parameters which is used to Aspect-Oriented Software Development. Boston:
generate a random value that is added to the value of /freq. Addison-Wesley, 2005.
[3] “Principles behind the agile manifesto,”
http://agilemanifesto.org/principles.html, accessed:
2014-03-29.
[4] G. Kiczales, J. Lamping, A. Mendhekar, C. Maeda,
C. Lopes, J. marc Loingtier, and J. Irwin, “Aspect-
oriented programming,” in ECOOP. SpringerVerlag,
1997.
[5] I. Akkaya, P. Derler, and E. Lee, “Aspect-oriented fault
modelling and anomaly detection in Ptolemy II.”
[6] D. Patterson and A. Fox, Engineering Software as a
Service: An Agile Approach Using Cloud Computing.
Strawberry Canyon LLC. Kindle Edition, 2014.
[7] M. Wright and A. Freed, “Open sound control: A
new protocol for communicating with sound synthe-
sizers,” in Proceedings of the International Computer
Music Conference, (Thessaloniki, Hellas), 1997, pp.
Figure 9: An aspect used to experiment with different 101–104.
probability distributions for smearing the spectrum.
[8] A. Freed, J. MacCallum, and A. Schmeder, “A dy-
namic, instance-based, object-oriented programming
in max/msp using open sound control message dele-
6. CONCLUSIONS AND FUTURE WORK gation,” in Proceedings of the International Computer
Music Conference, 2011.
We have presented an implementation of aspect-oriented
programming for data-flow languages such as Max/MSP
and PD, which can simplify a variety of tasks in arts/media
programming of an agile and speculative nature.

- 1591 -