A Web-Based Framework for Distributed Music System

Research and Creation

Benjamin Matuszewski

To cite this version:

Benjamin Matuszewski. A Web-Based Framework for Distributed Music System Research and Cre-
ation. AES - Journal of the Audio Engineering Society Audio-Accoustics-Application, 2020. �hal-
03033143�

HAL Id: hal-03033143

https://hal.science/hal-03033143
Submitted on 1 Dec 2020

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est

archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents
entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non,
lished or not. The documents may come from émanant des établissements d’enseignement et de
teaching and research institutions in France or recherche français ou étrangers, des laboratoires
abroad, or from public or private research centers. publics ou privés.
PAPERS

A Web-Based Framework for Distributed Music

System Research and Creation

Benjamin Matuszewski,
([email protected])

CICM/musidance EA1572, Université Paris 8

STMS Ircam-CNRS-Sorbonne Université
Paris, France

This paper presents soundworks, a framework dedicated to prototyping and developing

distributed multimedia applications using Web technologies. Since its first release in 2015,
the framework has been used in numerous artistic and research projects such as concerts,
installations, workshops, teaching or experimental setups. We first present how this diversity
of contexts and objectives permitted to identify a set of patterns able to support recurring needs
of expert users in exploratory tasks. We then detail new developments that have been achieved
to provide better support to these patterns. More particularly, we describe the novel distributed
state management system dedicated at simplifying the implementation of remote control and
monitoring interfaces and, the plug-in system implemented to improve the extensibility of
the framework and foster composition of dedicated functionalities. We believe that these new
developments can provide a solid ground to further research and artistic practices in the area of
distributed music systems. The soundworks framework is open-source and released under
BSD-3-Clause license.

0 INTRODUCTION sciences and humanities—and thus gather a great diversity

of goals, skills and methodologies (e.g. experimental stu-
The specification and development of the WebAudio dies, practices-based research). It appears that a common
API [1, 2]—alongside Application Programming Inter- ground for the support of this diversity can be found in
faces (API) such as WebSockets [3] or WebGL [4] and the concept of experimental systems—as systems compo-
the possibilities offered by a full-featured scripting lan- sed of epistemic things and technical objects in constant
guage such as JavaScript [5]—has permitted to envision evolution and reconfiguration—developed by Rheinberger
the Web platform [6] as a viable technical platform for ar- [16, 17] and pursued by Schwab in the context of artis-
tistic creation and more precisely for computer music prac- tic research [18]. We postulate that such epistemological
tices [7]. Furthermore, the recent developments of ubiqui- ground can lead to the implementation of particular pat-
tous and pervasive computing [8], with the democratiza- terns [19] in order to support this diversity of research
tion of smartphones and large spread of nanocomputers, practices effectively.
led to consider Web technologies as a possible solution for
recurring integration and interoperability issues [9]. These soundworks [20]—initiated by S. Robaszkiewicz and
two complementary aspects therefore authorize to consi- N. Schnell [21] in 2015—is a framework dedicated to
der the Web as an interesting environment in the deve- the development of distributed multimedia applications
lopment of Networked Music Systems [10, ?, 11]. Moreo- on the web. It has known two major revisions (in 2016
ver, this novel approach could unfold novel possibilities in and 2017) and has been used in numerous artistic and
related areas such as multi-source electro-acoustic music research projects (e.g. concerts, installations, workshops,
[12, 13] or interfaces for musical expression [14, 15]. In pedagogical or experimental setups) [22, 23, 24]. While
this context, the development of a dedicated framework, these achievements tended to validate the efficacy of the
designed to support both the specificities of the web plat- framework considered as an experimental platform, they
form and of computer music research and practices seems also permitted to highlight some inherent and recurring
essential. difficulties. The third version of soundworks—initiated
Indeed, computer music is a field that spans across mul- in 2019 [25]—presented in this paper aims to address some
tiple disciplines—from scientific to artistic through social of these difficulties, as well as to provide solid foundations

J. Audio Eng. Sco., Vol. 68, No. 10, author’s prepint, 2020 October 1
Matuszewski PAPERS

upon which environments facilitating the inclusion and 2.1 Concerts and Performances
agency of non-expert developers users can be built. Playground is an application that allows a composer /
performer to remotely distribute and control audio mate-
After a short review of the related works (cf. Section 1), rials rendered on the smartphones of the audience.
we describe in Section 2 different contexts in which our
framework has been successfully used in last two years,
and which informs us about recurring and important needs
our framework must support. In Section 3, we present an
overview of the framework architecture, philosophy and
basic functionalities. Finally, in Sections 4 and 5, we de-
tail new features dedicated at supporting patterns—namely
remote monitoring and control, composability and extensi-
bility—that we consider of primary importance to provide
an effective experimental platform supporting distributed
music systems research and creation.
F IGURE 1. On the left, Garth Paine performing Future Perfect.
On the right, screenshots of two of the four control interfaces of
1 RELATED WORKS the Playground application.
Max/MSP or Pure Data [26, 27] are well established
The application expose several dynamic control inter-
environments used since many years by artists and resear-
faces, optimized for touch interfaces such as tablets (see
chers in a wide range of contexts. The success of these
Figure 1), that can be jointly used :
visual programming environments lies in part in their suc-
— The first one allows for triggering sound files on a
cessful implementation of certain patterns that permitted
given smartphone represented on the screen as a co-
users to create and compose their own application in a
lored square.
very interactive fashion [24]. However, the environments
— The second one allows for controlling granular syn-
also come with their drawbacks in our context. First, there
thesis among subsets of the audience’s smartphones.
are not primarily oriented toward distributed applications
— The third one is dedicated at controlling the spa-
and are difficult to operate in large and dynamic networks
tial rendering of audio files synchronized among all
of computers. Second, they necessitate the installation of
smartphones.
a software, making applications difficult to distribute and
— Finally, the fourth one is dedicated at managing all
thus to deploy in large collective settings, precluding new
presets and configuration variables as well as at as-
forms of public and collective participation.
signing particular sound banks to the other control
interfaces.
On the Web platform, attempts have been made to im-
plement equivalent environments [28]. However, while in-
In this application, a number of strategies are implemen-
teresting, these tools are far for from being as mature as
ted to provide the composer and performer a dynamic envi-
the original ones. Also, they tends to neglect one of the
ronment in which they can test sonic material and configure
most interesting aspects of using the Web platform, na-
many aspect of the synthesis (e.g. dynamic update of sound
mely the network. Finally, some frameworks dedicated to
files, creation of presets) in the studio, but also have use-
network music systems, such as Rhizome [29] or Nexus,
ful feedback on the state of audience’s smartphones (e.g.
[30] have been proposed. While similar to soundworks
loading states, position in concert hall) during the perfor-
in their scope, these tools do not seem to be maintained or
mance.
in active development.
Playground has been designed together with the compo-
ser Garth Paine and implemented for the creation of Fu-
2 CONTEXTS ture Perfect, an immersive 3D audio visual performance 1 .
Since then, the application has been used for the crea-
In this section we review different contexts in which tion of several pieces—by the composer himself or other
we deployed web-based distributed systems implemented composers—, as well as in workshops and pedagogical si-
using our framework. Each of these contexts will be illus- tuations.
trated with a particular project that have been developed
in the last years. Note that while these 3 applications have 2.2 Installations
been designed and developed with the previous version of The context of an installation comes with different
soundworks, two of them (i.e. Playground and CoMo) constraints and requirements than the ones of perfor-
have already been ported to the novel version. These rewri-
tings permitted to simplify the code base, and moreover,
enabled new artistic and research possibilities, assessing 1. Future Perfect has been composed and realized during a re-
thus the concepts and design decisions presented in this search / creation residency that took place in 2018 between Ircam
paper. and ZKM.

2 J. Audio Eng. Sco., Vol. 68, No. 10, author’s prepint, 2020 October
PAPERS Soundworks

mances. In such contexts, the usage of so-called nano- CoMo has also been used in different settings such as
computers is interesting for several reasons [24], the most music, design and dance researches, artworks [36] and
important one being the simplicity they offer in term of workshops.
orchestration and tasks automations compared to smart-
phones. 2.4 Common Requirements and Patterns
These different examples show the large diversity of
contexts a framework dedicated at computer music re-
search and creation must support. Furthermore, they all
implied intertwined periods of research, development,
composition and tests in the laboratory or the studio (pos-
sibly with musicians and performers), that deepen further
the diversity of spaces and temporalities involved.

F IGURE 2. On the left, screenshot of the centralized control- To adapt to these different contexts and their inherent
ler developed for the installation Biotope composed by Jean-Luc constraints, the technological system must thus be easily
Hervé. On the right, session of measurement during the EmoDe- developed, modified or extended. This leads us to consider
mos research project. that two design aspects are of primary importance in the
development of our framework.
For example, Biotope [31], composed by Jean-Luc First, the importance for remote monitoring and control
Hervé 2 , is a generative and interactive installation that that allows a single user in working situation (e.g. compo-
features 27 Raspberry Pi nanocomputers running Node.js ser, researcher) to operate the distributed system—possibly
soundworks clients. The audio synthesis is achieved composed of hundreds of devices—as a “single coherent
using a Node.js wrapper on top of the libpd library [32, 33]. system” [37]. We will describe in Section 4 how our frame-
In this system, a number of strategies have been imple- work proposes to support and facilitate the implementation
mented to provide a dynamic and testable environment to of such functionality.
the composer and to the computer music designer. Among Second, the importance of being able to easily reuse
them, we have implemented a centralized controller dedi- existing functionalities but also to extend the framework
cated at controlling and monitoring the state and parame- with novel and dedicated components, to support explora-
ters of each agent in real-time. For example, each square tory workflows. Such problems can be addressed by intro-
in Figure 2 right, represents a musical agent in its relative ducing and supporting composability and extensibility in
position in the exhibition space, the different colors giving the system. We will describe in Section 5 how we propose
an overview of their state in real-time. to promote such aspects in soundworks.
2.3 Scientific Settings and Measurements
A third important context of computer music researches 3 ARCHITECTURE OVERVIEW
relates to scientific experimental research. In this context,
In this section, we present some high-level and general
some the characteristics of our framework such as clock
aspects of the soundworks framework. We present first
synchronization [34] enabled novel possibilities in scienti-
the general architecture and scope of the framework, and
fic experimentations.
second, a formalization of its most basic functionalities.
For example, the project EmoDemos [35] included an
experiment dedicated at measuring precision and synchro-
nization of the movement in groups of children practicing 3.1 General Principles
music (see Figure 2, right). This experimental setup has Since its inception, soundworks has been dedicated
been developed on top of CoMo—an application dedica- to simplifying the development of web-based and distribu-
ted at creating movement-based distributed Interactive Ma- ted real-time musical systems. Applications created using
chine Learning scenarios—[15], and allowed to record the soundworks follow a star network topology centered
motion sensors of smartphones tagged with synchronized around a server written using Node.js (see Figure 3). In
timestamps. The portability and simplicity of deployment these applications, clients can have multiple responsibili-
of the system permitted us to measure almost 200 children, ties (e.g. audio rendering, visual rendering, control) and be
divided by groups of 10 to 15. of different kinds (e.g. mobile, desktop, nanocomputers).
Once again, the system exposes a dedicated client to In previous versions, the framework was mainly focu-
control and monitor the state of the application, allowing sed on mobile applications and therefore privileged certain
the experimenters to prototype and refine the protocol as characteristics of these platforms (e.g. graphical user inter-
well as to ensure smooth measurements in a very constrai- face, usability). However, to support more diverse applica-
ned timeline and environment. tions and use-cases, it must evolve toward more modula-
rity and extensibility considering both software (e.g. inte-
2. Biotope has been realized at Ircam and created at the Centre gration of third party components and libraries) and hard-
Georges Pompidou, Paris in the context of the exhibition “La fa- ware (e.g. integration of IoT elements). In this objective,
brique du vivant” the scope of the framework has been refined and narro-

J. Audio Eng. Sco., Vol. 68, No. 10, author’s prepint, 2020 October 3
Matuszewski PAPERS

Figure 4 also illustrates a novel feature of the fra-

mework that enables the seamless implementation of
soundworks clients in the two main JavaScript en-
vironments : browsers and Node.js. Indeed, while this
approach has already been tested and deployed in a pro-
duction setting (cf. 2.2), the novel version the framework
properly integrates it by making most of the code compa-
tible to both platforms. This novel feature should foster
IoT approaches [38, 24] by simplifying the creation of
applications composed of diverse type of clients (e.g.
smartphones, nanocomputers).
F IGURE 3. Overview of the architecture of a typical
soundworks application. 4 DISTRIBUTED STATE MANAGEMENT

An important novel feature of soundworks is the in-

wed down to focus only on four key aspects : initialization, tegration of a distributed state management system. This
communications, distributed state management and plug-in component is dedicated to support and simplify the im-
host for external and dedicated functionalities. As a conse- plementation of remote control and monitoring functiona-
quence, a number of functionalities (e.g. templating, gra- lities.
phical and audio rendering) have been removed from the Since the introduction of the Flux pattern proposed by
core of the framework and delegated to external and spe- Facebook [39], usage of libraries that enforce unidirectio-
cialized libraries. These developments also permitted to re- nal and circular data flow in the application is considered
duce the API surface area of the framework, the number of a good practice among the JavaScript community. In our
dependencies, and finally improved its maintainability and case, using such pattern that consider rendering as a pure
learnability. function of the state, could therefore be very interesting,
as the state of any node could be modified from a remote
control interface in a transparent way for the node itself.
3.2 Initialization and Communications However, existing libraries are not firstly designed for dis-
The most basic functionality exposed by the frame- tributed applications and are difficult to adapt to our speci-
work, is to easily bootstrap an application by taking fic context for two main reasons. First, they do not forma-
care of initializing processes and communications. Figure lize nor integrate the notion of discrete and volatile events
4 summarizes the initialization process common to all very common in our applications (e.g. triggering a sound).
soundworks clients : Second, they do not provide a simple way to synchronize
— The init step consists in connecting two WebSo- states across several nodes in the network. To tackle these
ckets to the server, one dedicated to JSON compliant issues, we designed a novel component implementing such
string data and a second one to binary data. The API unidirectional and circular data flow approach, and adapted
of both sockets is similar and exposes a simple pu- to the particular requirements of our applications.
blish / subscribe interface.
— Once sockets are connected, the plug-ins initiali- 4.1 Concepts and Requirements
zation can start. To support dependencies between In our contexts, the application of such an unidirectional
plug-ins, soundworks can create a dependency and circular pattern presents certain particularities illustra-
graph start each plug-in accordingly. ted in Figure 5.
— Finally, when all plug-ins are in a ready state the
application specific code (called Experience in
soundworks’ terminology) can start.

F IGURE 5. Conceptual overview of a state management system

F IGURE 4. Initialization steps of a soundworks client, mobile enforcing unidirectional and circular data flow in a distributed
browser or Node.js process running on embedded hardware. context.

4 J. Audio Eng. Sco., Vol. 68, No. 10, author’s prepint, 2020 October
PAPERS Soundworks

First, the state of every client has to be kept synchro-

nized server-side. The rationale for this design strategy
stands in the importance of being able to remotely moni-
tor and control any client of the system from a centralized
point. Indeed, the possibility to dynamically interact with
any node of the network, and the rapid feedback loop it en-
ables, is of primary importance in working situations. Fur-
thermore, it appears to be crucial in exploratory contexts
(such as artistic and research activities) where the final ap-
plication cannot be specified beforehand and emerges from
an iterative process.
Second, Figure 5 highlights the need of a certain gra-
nularity in the definition and synchronization of the states.
More precisely, while some variables and parameters (na-
med globals in Figure 5) needs to be accessible to every
client (e.g. master volume, mute), the particular state a
client (clients[2] in Figure 5) should not be shared
with all its peers. It only needs to be monitored or control-
led by particular types of clients dedicated to authoring and
/ or performance situations.

4.2 Protocol and API

F IGURE 6. Overview of the protocol designed for the
To fulfill these requirement while preserving the idea of soundworks state management system.
unidirectional and circular flow between actions, data and
rendering, we designed a simple protocol and implemented
a new component. The main principles of the protocol we 1
2
// SERVER-SIDE
const synthSchema = {
propose are : 3
4
volume: { type: 'float', min: -80, max: 6 },
trigger: { type: 'any', event: true },
— Allow any node to create a new state from a declared 5
6
};
stateManager.registerSchema('synth', synthSchema);
schema. 7
8 // CLIENT-SIDE
— Allow to keep the state synchronized with the server. 9
10
const playerState = await stateManager.create('synth');
playerState.subscribe(updates => {
— Allow any node to observe new states created on the 11
12
for (let [key, val] of Object.entries(updates)) {
if (key === 'volume') {
network. 13
14
mixer.volume = val;
} else if (key === 'trigger') {
— Allow any node to attach to a state created by ano- 15
16 }
synth.trigger();

ther node. 17
18
}
});
19 // ...later (or from any other attached node)
20 playerState.set({ volume: -6 });
Figure 6 illustrates a generic scenario enabled by this
protocol. A client (named controller) observes the ser-
F IGURE 7. Pseudo-code - Main aspects of the soundworks
ver and attach to the state created by another client (na- state manager API.
med player). Once attached, the controller receives
a notification each time the state is updated by its creator
(or any other attached node), enabling remote monitoring. The simplicity of these synchronized data structures also
The controller can also update values of the attached state, enables more advanced uses of dynamic composition of
enabling remote control. At any moment, the controller can states or distributed hierarchical state machines.
detach from the state and stop receiving update notifica-
tions 3 .
The protocol is abstracted behind a reduced API illus- 5 A HOST FOR PLUG-INS
trated in the pseudo-code example of Figure 7. This simple
example also highlights two interesting aspects of the com- Another important evolution of soundworks lies in its
ponent : ability to act as a plug-in host for extending its basic func-
— The complete abstraction of network communica- tionalities. We believe this feature will also to enhance mo-
tions, allowing users to focus on the application logic dularity, allowing to combine predefined components for a
rather than routing of network messages. specific application, but also to simplify maintenance and
— The possibility to use schemas declarations to gene- evolutions of both the framework and the applications. A
rate controls and monitoring interfaces, simplifying number of plug-ins dedicated at synchronizing clocks, re-
fast prototyping and testing of ideas as well as im- cording data, parsing and watching the file system, to name
plementation of dynamic and complex interfaces. a few, are already available.
In this section, we first present a technical overview of
3. Note that no particular guard has been implemented to the implementation and registration of a soundworks
prevent race conditions, therefore the last event received wins. plug-in. Second, we illustrate this feature with two novel

J. Audio Eng. Sco., Vol. 68, No. 10, author’s prepint, 2020 October 5
 Matuszewski PAPERS

components dedicated at runtime distributed scripting and 5.2 Examples

logging of arbitrary data. To illustrate the kind of functionalities the plug-in host
system enable, we present two plug-ins we created for
5.1 Implementing and Registering Plug-ins the novel version of the CoMo application (cf. Section
Thanks to the dynamic nature of the JavaScript lan- 2.3). While designed and implemented with this specific
guage, the implementation of a new plug-in is relatively use-case in mind, these two examples stands to be good
simple. examples of how this architecture facilitate the creation of
modular and reusable components.
1 // export a factory function
2 export default function pluginFactory(AbstractPlugin) {
3 return class DelayPlugin extends AbstractPlugin { 5.2.1 Runtime Distributed Scripting
4 constructor(client, name, options) {
5
6
super(client, name);
this.options = this.configure({ delayTime: 1 }, options);
The first plug-in we present, illustrated in Figure 10 is
7
8
} dedicated at the scripting of focused parts of the applica-
9
10
start() {
super.start();
tion at runtime 4 . As such, the plug-in seeks to simplify the
11
12
// emulate asynchronous bootstrapping task 1
setTimeout(() => {
test of ideas and strategies (e.g. mappings, audio synthesis)
13
14
// notify manager that the plugin is started
this.started();
in a very efficient manner : without having to reload the
15
16
// emulate asynchronous starting task 2
setTimeout(() => {
whole application—server and / or clients—nor having to
17
18
// notify manager that the plugin is ready
this.ready();
implement each time a dedicated control interface.
19 }, this.options.delayTime * 1000);
20 }, Math.random() * 1000);
21 }
22 }
23 }

F IGURE 8. Pseudo-code - Main aspects of the implementation of

a soundworks plug-in.

Figure 8 illustrates several important aspects of the im-

plementation of a new plug-in. First, the module exports
a factory function that itself returns the plug-in class de-
finition. This simple pattern allows soundworks to dy-
namically pass the AbstractPlugin parent class to the
plug-in factory function and thus avoid hard-coded and cir-
cular dependencies between the plug-in and the host. Se-
cond, it shows (cf. start method) the different states that F IGURE 10. Screenshot of the runtime distributed scripting in-
terfaces. The function written on the editor (right) is dynamically
the plug-in must report to the host. Indeed, reporting these executed on the two other clients (left) when updated.
steps are important to be able to deal with all the different
asynchronous tasks that has to be performed (e.g. network Additionally, we think this plug-in can play an important
communication, particular GUI and user interactions) du- pedagogical role by providing to users without expert pro-
ring the initialization of the application. gramming knowledge (e.g. researcher, composers), a focu-
sed entry point where they can work within their own do-
1
2
import { Server } from '@soundworks/core/server';
import delayPluginFactory from '@soundworks/plugin-delay/server';
main of expertise without having to understand the whole
3
4 const server = new Server();
code base and architecture.
// override the default `delay` option
5
6 server.registerPlugin('delay-1', delayPluginFactory, { delay: 2 });
We believe this functionality may turn out as an impor-
7
8
// declare that 'delay-2' must wait for 'delay-1'
// to be ready before starting itself
tant addition to the tools our framework provide to support
9 server.registerPlugin('delay-2', delayPluginFactory, {}, ['delay-1']);
rapid prototyping, exploration and testing of ideas.

F IGURE 9. Pseudo-code - Server-side configuration and registra- 5.2.2 Logging and Data Recording
tion of a plug-in into soundworks. The second component we present is dedicated to log-
ging and storing on the server, arbitrary data produced by
Figure 9 illustrates how a plug-in is registered into
any node of the network 5 . Indeed, simplifying access to
soundworks (while Figure 9 shows the process server-
such functionality to record and analyze data is obviously
side, similar code would be written client-side) as well
central to many scientific and research practices.
as two other possibilities. First, the possibility for a given
However, we believe that the simplicity of usage illus-
plug-in factory to be used several times by registering it
trated in the Figure 11 will also help to develop usages in
with a different identifier (e.g. delay-1 and delay-2).
other directions. For example, for auditing the system, tes-
For example, this capacity could be used to synchronize
ting components or benchmarking concurrent implemen-
different clocks (e.g. audio clock and high precision clock)
on the same client. Second, it shows how dependencies bet-
ween several plug-ins can be declared, enabling the possi- 4. https://github.com/collective-soundworks/
soundworks-plugin-scripting
bility of implementing higher-order plug-ins on top of the 5. https://github.com/collective-soundworks/
functionalities offered by lower-level ones. soundworks-plugin-logger

6 J. Audio Eng. Sco., Vol. 68, No. 10, author’s prepint, 2020 October
 PAPERS Soundworks

1 const pathname = `${date}-${uuid}/${username}.csv`;

also been supported by the Ircam project BeCoMe, which is
2
3
const log = await loggerPlugin.create(pathname); featured in the Constella(c)tions residency of the STARTS
4
5
// later
log.write(`${time}; ${x}; ${y};`);
program of the European Commission.
We would like to thank our projects partners and our col-
leagues at IRCAM for their precious contributions to the
F IGURE 11. Pseudo-code - Creation of a ‘csv‘ log file and wri-
ting of arbitrary data using the logger plug-in.
project.

8 Bibliographie
tations in real-world situations, or, for recording and re-
playing examples of interactions (e.g. sensor data) to work [1] “WebAudio API Specification,” URL https://
on mappings and audio synthesis in the studio. www.w3.org/TR/webaudio/.
[2] H. Choi, “AudioWorklet : The future of web audio,”
6 CONCLUSION AND FUTURE WORKS presented at the Proceedings of the International Computer
Music Conference, p. 7.
In this paper, we have presented the motivations, de- [3] “The WebSocket Protocol,” URL https://
sign and implementation aspects of the novel version of tools.ietf.org/html/rfc6455.
soundworks, a framework dedicated at developing dis- [4] “WebGL Specification,” URL https://
tributed multimedia applications on the web. First, we have www.khronos.org/registry/webgl/specs/
presented the different contexts such a framework should latest/.
support, and illustrated these contexts with three projects [5] A. Wirfs-Brock, B. Eich, “JavaScript : the first 20
we developed last few years. These different contexts al- years,” Proceedings of the ACM on Programming Lan-
lowed us to show that supporting particular patterns is guages, vol. 4, pp. 77 :1–77 :189, doi :https://doi.org/10.
important for exploratory tasks. We then presented the 1145/3386327.
general architecture and two novel features of our fra- [6] T. Berners-Lee, R. Cailliau, J. Groff, B. Pollermann,
mework : 1. the distributed state management system, “World-Wide Web : The Information Universe,” Internet
dedicated at simplifying the implementation of remote Research, vol. 20, no. 4, pp. 461–471, doi :https://doi.org/
control and monitoring, and 2. its capacity to host external 10.1108/10662241011059471, initially published in Elec-
plug-ins, to foster composability and extensibility. tronic Networking, vol.2, no.1, Spring 1992.
[7] L. Wyse, S. Subramanian, “The viability of the web
While we think this novel version of soundworks pro- browser as a computer music platform,” Computer Music
vides solid foundations to further explore the possibilities Journal, vol. 37, no. 4, pp. 10–23, doi :https://doi.org/10.
of the web platform in the area of distributed music sys- 1162/COMJ\ a\ 00213.
tems, it also opens new questions and large areas for new [8] M. Weiser, “The Computer for the 21st Century,”
developments. An important aspect that needs to be recon- ACM SIGMOBILE Mobile Computing and Communica-
sidered and solved is the interoperability between the fra- tions Review, vol. 3, no. 3, pp. 3–11, doi :https://doi.org/10.
mework and other tools, such as graphical or audio libra- 1145/329124.329126, initially published in Scientific ame-
ries. In this regard, we think that while the schema format rican, vol. 265, no.3, 1991.
used for the state management component could provide a [9] D. Guinard, V. Trifa, “Towards the Web of Things :
good basis in that direction, it is for now insufficiently spe- Web Mashups for Embedded Devices,” presented at the In
cified. Another important limitation and direction of im- MEM 2009 in Proceedings of WWW 2009. ACM, p. 8.
provement is the lack of support for collections in the state [10] S. Gresham-Lancaster, “The Aesthetics and His-
management system, such addition would facilitate the im- tory of the Hub : The Effects of Changing Technology
plementation of advanced features such as presets or sound on Network Computer Music,” Leonardo Music Journal,
banks. Finally, to further simplify and fasten the implemen- vol. 8, pp. 39–44, doi :https://doi.org/10.2307/1513398.
tation of new applications, a Command Line Interface tool [11] G. Weinberg, “Interconnected Musical Networks :
for scaffolding components, clients or plug-ins would be Toward a Theoretical Framework,” Computer Music Jour-
an important addition. We believe that the addition of these nal, vol. 29, no. 2, pp. 23–39, doi :https://doi.org/10.1162/
features could foster further research and artistic practices 0148926054094350.
and maybe provide a common ground for pluralistic ap- [12] F. Bayle, “Space, and more,” Organised Sound,
proaches in the area of distributed music systems. vol. 12, no. 3, pp. 241–249, doi :https://doi.org/10.1017/
S1355771807001872.
7 ACKNOWLEDGMENT [13] B. Taylor, “A History of the Audience as a Spea-
ker Array,” presented at the Proceedings of the NIME’17
The presented work has been initiated in the CoSiMa Conference (2017).
research project funded by the French National Research [14] I. Poupyrev, M. J. Lyons, S. Fels, T. Blaine (Bean),
Agency (ANR, ANR-13-CORD- 0010) and further deve- “New Interfaces for Musical Expression,” presented at the
loped in the framework of the Rapid-Mix Project from the CHI ’01 Extended Abstracts on Human Factors in Com-
European Union’s Horizon 2020 research and innovation puting Systems, CHI EA ’01, pp. 491–492, doi :https:
program (H2020-ICT-2014-1, Project ID 644862). It has //doi.org/10.1145/634067.634348.

J. Audio Eng. Sco., Vol. 68, No. 10, author’s prepint, 2020 October 7
Matuszewski PAPERS

[15] B. Matuszewski, J. Larralde, F. Bevilacqua, “Desi- [26] M. Puckette, “Combining Event and Signal Pro-
gning Movement Driven Audio Applications Using a Web- cessing in the MAX Graphical Programming Environ-
Based Interactive Machine Learning Toolkit,” presented at ment,” Computer Music Journal, vol. 15, no. 3, pp. 68–77,
the Proceedings of the 4th Web Audio Conference (2018). doi :https://doi.org/10.2307/3680767, publisher : The MIT
[16] H.-J. Reinberger, “Experimental Systems : Histo- Press.
riality, Narration, and Deconstruction,” Science in Context, [27] M. Puckette, “A case study in software for artists :
vol. 7, no. 1, pp. 65–81, doi :https://doi.org/10.1017/ Max/MSP and Pd,” in Art++ (David-Olivier Lartigaud),
S0269889700001599, publisher : Cambridge University hyx ed. (2016).
Press. [28] “WebAudio-Patcher,” URL https://github.
[17] H.-J. Rheinberger, “Consistency from the pers- com/Fr0stbyteR/webaudio-patcher.
pective of an experimental systems approach to the [29] S. Piquemal, “Rhizome,” URL https://
sciences and their epistemic objects,” Manuscrito, github.com/sebpiq/rhizome.
vol. 34, no. 1, pp. 307–321, doi :https://doi.org/10.1590/ [30] J. Allison, Y. Oh, B. Taylor, “NEXUS : Collabo-
S0100-60452011000100014. rative Performance for the Masses, Handling Instrument
[18] M. Schwab, Experimental Systems : Future Know- Interface Distribution through the Web,” presented at the
ledge in Artistic Research, Orpheus Institute series (Leuven Proceedings of the NIME’13 Conference (2013).
University Press). [31] “Biotope Presentation,” URL https://youtu.
[19] C. Alexander, Notes on the Synthesis of Form (Har- be/RmSujqdT6L0.
vard University Press). [32] P. Brinkmann, C. McCormick, P. Kirn, M. Roth,
[20] “Soundworks Repository,” URL https: R. Lawler, H.-C. Steiner, “Embedding Pure Data with
//github.com/collective-soundworks/ libpd,” presented at the Pure Data Convention Weimar
soundworks. 2011.
[21] S. Robaszkiewicz, N. Schnell, “Soundworks – a [33] “Node-libpd Repository,” URL https://
playground for artists and developers to create collabora- github.com/ircam-jstools/node-libpd.
tive mobile web performances,” presented at the Procee- [34] J.-P. Lambert, S. Robaszkiewicz, N. Schnell, “Syn-
dings of the 1rst Web Audio Conference (2015). chronisation for Distributed Audio Rendering over Hete-
[22] N. Schnell, B. Matuszewski, J.-P. Lambert, S. Ro- rogeneous Devices, in HTML5,” presented at the Procee-
baszkiewicz, O. Mubarak, D. Cunin, S. Bianchini, X. Bois- dings of the 2nd Web Audio Conference.
sarie, G. Cieslik, “Collective Loops : Multimodal Interac- [35] “Emodemos Website,” URL https://www.
tions Through Co-located Mobile Devices and Synchro- unige.ch/cisa/emodemos/.
nized Audiovisual Rendering Based on Web Standards,” [36] “Constella(c)tions - Residency,” URL https://
presented at the Proceedings of the Tenth International vertigo.starts.eu/calls/starts-residencies-call-
Conference on Tangible, Embedded, and Embodied Inter- residencies/constellactions/detail/.
action, pp. 217–224, doi :https://doi.org/10.1145/3024969. [37] M. van Steen, A. S. Tanenbaum, “A brief in-
3024972. troduction to distributed systems,” Computing, vol. 98,
[23] B. Matuszewski, N. Schnell, F. Bevilacqua, “In- no. 10, pp. 967–1009, doi :https://doi.org/10.1007/
teraction Topologies in Mobile-Based Situated Networ- s00607-016-0508-7.
ked Music Systems,” Wireless Communications and Mo- [38] L. Turchet, C. Fischione, G. Essl, D. Keller,
bile Computing, vol. 2019, pp. 1–9, doi :https://doi.org/10. M. Barthet, “Internet of Musical Things : Vision and
1155/2019/9142490. Challenges,” IEEE Access, vol. 6, pp. 61994–62017, doi :
[24] B. Matuszewski, F. Bevilacqua, “Toward a Web of https://doi.org/10.1109/ACCESS.2018.2872625.
Audio Things,” presented at the Proceedings of the 15th [39] “Flux Pattern,” URL https://facebook.
Sound and Music Computing Conference (2018). github.io/flux/.
[25] B. Matuszewski, “Soundworks A Framework for
Networked Music Systems on the Web,” presented at the
Proceedings of the 5th Web Audio Conference, p. 6.

THE AUTHORS

8 J. Audio Eng. Sco., Vol. 68, No. 10, author’s prepint, 2020 October